Ripping HDCDs: Best practices?

[tewill]

Re: Ripping HDCDs: Best practices?

- Remasters are often louder. Older albums mastered with headroom, that's often the "non-remasters".

If there is no PE, the signal could be left as-is.

When test-ripping some of my HDCDs, I found that some for which PE was disabled were still mastered on the loud side (many of which weren't remasters) and showed clipping when the HDCD wasn't decoded. I was previously thinking that HDCDs without PE enabled might not need to be decoded, but after seeing this it made me consider the possibility that some HDCDs with PE disabled might be purposely mastered on the louder side with the assumption that their volume would be reduced by 6 dB when played (because they'd be decoded). Point being I'm not sure if it can be said that across the board it is unnecessary to decode HDCDs with PE disabled, but this is mainly only a concern to begin with if you play on devices that can't decode HDCD or if you are transcoding. Regardless, it's still a bit of a pain, as in such cases it could mean you'd need to do the extra work of seeing whether an HDCD with PE disabled was mastered on the louder side (and decode the HDCD, but don't add any amplification) or mastered on the quieter side and not decode the HDCD (or decode it using the HDCD DSP and either add the +6 dB amplification or write ReplayGain tags if you use ReplayGain).

The latter is a bit ambiguous - what's this "this" which is meaningless? It is not meaningless to decode a signal which uses PE, as it reverses compression. It is of course meaningless to try to stuff the last eight bits down the throat of a DAC that can only chew 16.

This is a direct quote from the referenced post. I took it to mean your latter point.

[Porcus]

Re: Ripping HDCDs: Best practices?

When test-ripping some of my HDCDs, I found that some for which PE was disabled were still mastered on the loud side (many of which weren't remasters) and showed clipping when the HDCD wasn't decoded.

That should not happen. Either there has been some decoding done (and the processing has sent the signal beyond 0dBFS), or they simply sound that way when not decoded. (Since it then is dynamically compressed, that is maybe what you hear.)

or write ReplayGain tags if you use ReplayGain).

Did you use RG on those files? Checked your settings? (If you use fb2k: prevent clipping according to peak? Amplified those which are not RG'ed?)

[tewill]

Re: Ripping HDCDs: Best practices?

That should not happen. Either there has been some decoding done (and the processing has sent the signal beyond 0dBFS), or they simply sound that way when not decoded. (Since it then is dynamically compressed, that is maybe what you hear.)

I checked again, and I am mistaken - there isn't clipping in my un-decoded lossless test tracks, but there is clipping when I transcode the un-decoded lossless files to mp3 as well as if I first apply the HDCD DSP with the +6 dB amplification and then transcode to mp3 (PE was disabled for all tracks). So it appears my point is only relevant when transcoding, i.e. when transcoding HDCD-encoded files to mp3, if they're mastered on the louder side they might show clipping and you might want to consider using the HDCD DSP without the +6 dB amplification prior to transcoding.

So, still a small pain if you're transcoding, as it seems you can't take a blanket approach (e.g. apply the HDCD DSP prior to transcoding only if PE is enabled or always apply the HDCD DSP prior to transcoding, but disable the +6 dB amplification), as I've also seen some transcoded tracks for which I first applied the HDCD DSP without the +6 dB amplification and the resulting mp3s were prohibitively quiet (i.e. even with the volume all the way up they were still too quiet).

Did you use RG on those files? Checked your settings? (If you use fb2k: prevent clipping according to peak? Amplified those which are not RG'ed?)

I don't use ReplayGain, and haven't used any player settings to correct clipping; I'm seeing the clipping by looking at the waveform in Audacity.

BTW, all this probably really is pedantic [for me] because for the most part I have difficulty hearing differences between HDCD tracks with different treatments. Besides noticing the obvious volume decrease when applying the HDCD DSP without amplification, the only other difference I've noticed was that when transcoding to mp3 from tracks with PE enabled, the mp3s did seem to sound better if the HDCD DSP was applied without the +6 dB first, as opposed to transcoding without the HDCD DSP. But even then I really had to strain to hear a difference. I also have trouble hearing issues caused by the clipping I mentioned in the transcoded mp3s, so my guess is that after all this detail work I'll end up with a pretty simplistic approach to dealing with HDCDs.

Thanks for the help though!

[Mike48]

Re: Ripping HDCDs: Best practices?

tewill -- Thanks for the detailed summary. I appreciate it very much -- it's quite useful -- and I have in fact changed the defaults I use when decoding.

[tewill]

Re: Ripping HDCDs: Best practices?

Circling back on this, here are the best practices I came up with for ripping HDCDs:

1. In general, rip HDCDs to a lossless format and don't apply the HDCD DSP.
   
   1. You also shouldn't write ReplayGain tags on these lossless files because:
      
      1. ReplayGain can negatively affect the PE and LLE features.
         
         
      2. When you use the ReplayGain DSP, values are calculated based on the un-decoded HDCD audio, and these values won't be valid once the files are decoded by your device.
   
   
2. That said, if you know for a fact you'll be playing these lossless files on devices that can't decode HDCD-encoded files:
   
   1. Use foobar2000 to analyze the files and see which HDCD features are enabled.
      
      
   2. If Peak Extend is enabled, apply the HDCD DSP to the files, with the +6 db amplification option *unchecked*, and don't write ReplayGain tags.
      
      
   3. If Peak Extend is not enabled, leave the files as they are and do not apply the HDCD DSP.
      
      
   4. NOTE: Unless you don't mind the possibility of re-ripping CDs in the future, if you choose to apply the HDCD DSP to your lossless files consider first making archive copies, as the HDCD DSP will physically alter the audio data.
   
   
3. If you plan on transcoding the lossless files to a lossy format:
   
   1. Same as above, if PE is enabled apply the HDCD DSP while transcoding, with the +6 db amplification option *unchecked*. Otherwise don't apply the HDCD DSP while transcoding.
      
      1. NOTE: Applying the HDCD DSP *while* transcoding doesn't appear to affect the source lossless files. You could also apply the HDCD DSP to the lossless files prior to transcoding, but again note that doing so will physically alter the audio data in those files.

[EliC]

Re: Ripping HDCDs: Best practices?

I finally stopped using the HDCD DSP. As an added benefit, my CD ripping has increased probably >10x. Something about the implementation of that DSP really slows down ripping.

Now I need to figure out which discs are HDCDs and go back and re-rip them...

[EliC]

Re: Ripping HDCDs: Best practices?

When transcoding, I like to apply replaygain to my lossy files. Is there a one step way to detect HDCD, apply the HDCD DSP, and then apply replaygain?

[tewill]

Re: Ripping HDCDs: Best practices?

Now I need to figure out which discs are HDCDs and go back and re-rip them...

Just in case you don't already know this, you can use dBpoweramp for this purpose, by using it to see what the sample size is for lossless files to which you applied the HDCD DSP. If they're flagged as 24 bit, they were ripped from HDCDs. It's still a bit of a pain, but I find using the Batch Converter can be a bit easier: once added to the Batch Converter, you can select "List" from the menu then add "Sample Size" as a column if it isn't already there.

When transcoding, I like to apply replaygain to my lossy files. Is there a one step way to detect HDCD, apply the HDCD DSP, and then apply replaygain?

A few things here: First, from what I've found, when transcoding to lossy you should only apply the HDCD DSP if the PE feature is enabled, and, if it is, you shouldn't use ReplayGain because it could affect the benefits of peak extension. Yes, if you're transcoding from HDCD-encoded lossless audio the resulting lossy audio will be a bit quieter, but it won't be by much (especially relative to those transcoded from HDCD-encoded files that don't have PE enabled) and the benefit is that it should sound better than it would if the HDCD wasn't decoded. That said, if it is an issue that they're quieter, it might not be worthwhile to decode the HDCD to begin with. Second, I have found that you can apply the HDCD DSP while transcoding and it will work to decode HDCD audio for the lossy files and not affect the audio in the lossless source files. However, if you still wanted to apply the ReplayGain DSP, from what I've found you can't do so during transcoding, as it will base its values upon the un-decoded audio; so you therefore would have to apply the ReplayGain DSP as part of a second step, after the files are transcoded to a lossy format.

[EliC]

Re: Ripping HDCDs: Best practices?

Spoon, any chance we can have these options build into the HDCD DSP?

- Only apply HDCD DSP if:
( ) Peak Extend (PE) is enabled
( ) Low-Level Extension (LLE) is enabled
( ) Transient Filter (TF) is enabled

- Apply HDCD DSP, recalculate replaygain/EBU R128, then apply

[garym]

Re: Ripping HDCDs: Best practices?

Quick (I think, or maybe just naive) question. If I have an HDCD that was ripped to FLAC without use of the HDCD dsp, is it true that I can:

1. convert from FLAC to FLAC, adding the HDCD DSP in this conversion, and not *also* applying ReplayGain
2. Afterwards apply ReplayGain to the newly created FLAC files

I realize that these new FLAC files would no longer be lossless, but I'd be able to play them with non-HDCD players. And to be clear, if I play my original FLAC files (that were NOT ripped using HDCD DSP) through a player that is capable of handling HDCD decoding, they should play as HDCD disks.

[garym]

Re: Ripping HDCDs: Best practices?

Quick (I think, or maybe just naive) question. If I have an HDCD that was ripped to FLAC without use of the HDCD dsp, is it true that I can:

1. convert from FLAC to FLAC, adding the HDCD DSP in this conversion, and not *also* applying ReplayGain
2. Afterwards apply ReplayGain to the newly created FLAC files

That is, it is NOT important that I apply HDCD DSP during *ripping* as opposed to simply applying it during a FLAC to FLAC conversion.

[tewill]

Re: Ripping HDCDs: Best practices?

Quick (I think, or maybe just naive) question. If I have an HDCD that was ripped to FLAC without use of the HDCD dsp, is it true that I can:

1. convert from FLAC to FLAC, adding the HDCD DSP in this conversion, and not *also* applying ReplayGain

Yes, this is what I would recommend.

2. Afterwards apply ReplayGain to the newly created FLAC files

And yes again. Reason being that if you apply the ReplayGain DSP at the same time as the HDCD DSP, the calculated ReplayGain values are based on the un-decoded HDCD audio. When you apply the ReplayGain DSP on the newly created FLAC files, the values are calculated based on the decoded HDCD audio. That said, I'd still recommend you not use ReplayGain for either HDCD-encoded or decoded files, as it might negatively affect the benefits of HDCD encoding (e.g., The PE feature extends the peaks, and decoding the HDCD audio decreases the volume by 6 dB to add more headroom, to ensure you both can hear this as well as to prevent clipping. As the purpose of ReplayGain is to tell a player to increase or lower the volume depending on the relative volume of a track, it can therefore affect decoded HDCD audio. Without ReplayGain, the volume of a decoded HDCD track will be a bit lower, but this won't be as noticeable with PE enabled, and hopefully you'll be paying more attention to how much better it sounds than the fact you might have to turn the volume up a bit).

I realize that these new FLAC files would no longer be lossless, but I'd be able to play them with non-HDCD players. And to be clear, if I play my original FLAC files (that were NOT ripped using HDCD DSP) through a player that is capable of handling HDCD decoding, they should play as HDCD disks.

The new FLAC files are still lossless, but the volume has been decreased by 6 dB. And, unlike the ReplayGain DSP, this volume change was effected by altering the audio in the files, so even though the files are still lossless (in that audio data was not lost), they have been permanently modified. And, you should still be able to play HDCD-encoded files on players that can't decode HDCD, as HDCDs were designed to be backwards-compatible; in such a case you'll hear the music but the HDCD will not be decoded. Last, yes, if you play HDCD-encoded files with a player that can decode HDCD, the HDCD should be decoded. I like foobar2000 because with the HDCD decoder add-in you can not only play HDCDs but configure the UI to show you both that the file is HDCD-encoded and with which features.

That is, it is NOT important that I apply HDCD DSP during *ripping* as opposed to simply applying it during a FLAC to FLAC conversion.

I'd say your "NOT" is misplaced here, as I'd instead say it is important that you NOT apply the HDCD DSP during ripping because it physically modifies the file, unless of course you don't want an exact copy of the HDCD or don't mind re-ripping in the future.

[garym]

Re: Ripping HDCDs: Best practices?

Thanks tewill. This is very clear. Thanks again for all the detail you've provided in this thread.

[JustaNewUser]

Re: Ripping HDCDs: Best practices?

Hi,
thanks for all this info in these posts but correct me if I am wrong:
if you rip *without* the HDCD option the lossless flac file will be 16 bit and no more; thus the extra bits of a HDCD would be lost. So how could you later transcode or play back such a file and expect that a HDCD capable player would somehow restore those lost bits that are not there? I thought the HDCD option ensures that the flac will be of a higher bitrate capable of storing the extra data.

If my assumptions are correct, let me explain my thoughts/questions regarding the HDCD option:
What do we want when we rip a CD? The best sounding, preferably lossless, most accurate file that resembles the original, we can get.

From what I understood the options regarding HDCD disks which make sense would be to either:
1. Rip to a lossless (ex. flac) file with the option of HDCD with +6 amplification.
2. Rip to a lossless file with the option HDCD without the amplification.
3. Rip to a lossless file *without* the HDCD option.

1. Could result in some clipping which is hard to discern, but has (slightly and not audiable) altered (amplified) the data.
2. Would result in a file with less volume.
3. Would result in a file that misses/discards the extra data of the HDCD thus resulting in a file that is no longer lossless.

Now correct me again but it seems that I would rather take a slight chance of a hard to discern change (clipping) than have a less than lossless file. So option 1 would be better than 3.
Since we all have a volume control, I think option 2 would even be better: no clipping and only a little quiter which we all can correct easily. (And to those that say my mobile player can't play those files at an adaquate volume, I would reccomend quality headphones and possibly a portable amp, not that listening to music on the street requires a super hi-def lossless file anyhow)

Regardless, the ReplayGain can always be calculated at a later time (by almost any modern program) and increase (or decrease) volume accordingly. You may get a wrong gain value when using this option in conjunction with the HDCD option, at the time of the rip, but that can always be corrected by recalculating the value (no loss of data or change in quality). Then almost all media players (including the oh so popular itunes) can increase the volume of the HDCD rips that have a lower volume, with no damage to the integrity of the lossless rip.

So in summary: I think ripping *without* the +6 ampl. but with the HDCD would probably result in the best quality, with the only caveat of it being lower in volume but with no clipping, distortion or loss of data. And to counter that, use the volume knob or/and replay gain.
This would probably be the easiest and most compatible option for fast ripping, just leave it on, get all the data and lossless files.

I hope this is somewhat correct.

I have a question though: when I leave the option HDCD on while I rip non HDCD discs, will that change anything at all in the files for those discs?

Also: when ripping with the HDCD option and creating both flac and mp3 files at the same time, would it then be better to use amplification specifically for the mp3s? I mean with replaygain I suppose you could just get those up in volume as well..

Thanks for your clarification on this.

[EliC]

Re: Ripping HDCDs: Best practices?

Hi,
thanks for all this info in these posts but correct me if I am wrong:
if you rip *without* the HDCD option the lossless flac file will be 16 bit and no more; thus the extra bits of a HDCD would be lost.

You are wrong. The info is all still there, and the original 16 bit PCM files ripped from the CD, stored in WAV and compressed with lossless compression (FLAC) can still later be run through the HDCD DSP.

You can verify this yourself.

Rip an HDCD with and without the HDCD DSP

Use the DSP on the one ripped without and see that they are they same.

This is basically what dB does anyway.

The better (best for lossless) practice is as stated, rip WITHOUT the HDCD dsp!