Varying encoder speeds?

[jsa307]

I'm seeing some very different encoder speeds, depending on file type and whether the batch encoder is used-- which I just wanted to verify is normal.

I'm new to using the encoder, but my few prior uses (only converting a single file type to another format) have been very fast, with encoding speeds > 150x. I just used the batch encoder on a number of FLAC files, using the multi-encoder to output to both ALAC and MP3 files. This took over 40 mins, with the encoder work at about 4x. While I understand that different files types may require more or less resources to manipulate, the difference in encoding speed for this total batch conversion is striking. As a test, I then took one of the individual FLAC files from the batch conversion and generated a WAV file. The encoding speed for this was 210x. Then I tried an idividual WAV -> ALAC conversion (65x speed) and an individual MP3 conversion (30x speed) from my WAV file.

It seems as if for some reason, using the batch converter is slowing things down? The individual steps, when performed manually, seem to add up to far less time than the batch converter took to handle one the files. Or am I missing a key point? Thanks!

[timw0932]

Re: Varying encoder speeds?

I'm seeing some very different encoder speeds, depending on file type and whether the batch encoder is used-- which I just wanted to verify is normal.

I'm new to using the encoder, but my few prior uses (only converting a single file type to another format) have been very fast, with encoding speeds > 150x. I just used the batch encoder on a number of FLAC files, using the multi-encoder to output to both ALAC and MP3 files. This took over 40 mins, with the encoder work at about 4x. While I understand that different files types may require more or less resources to manipulate, the difference in encoding speed for this total batch conversion is striking. As a test, I then took one of the individual FLAC files from the batch conversion and generated a WAV file. The encoding speed for this was 210x. Then I tried an idividual WAV -> ALAC conversion (65x speed) and an individual MP3 conversion (30x speed) from my WAV file.

It seems as if for some reason, using the batch converter is slowing things down? The individual steps, when performed manually, seem to add up to far less time than the batch converter took to handle one the files. Or am I missing a key point? Thanks!

I will guess the source and destination drives are the same, and in addition to the multiple outputs, you're using the multiple CPU feature. This creates an extreme I/O bound situation, and it essentially turns your conversion into a random I/O disk benchmark. For more, see my threads:

http://forum.dbpoweramp.com/showthread.php?t=22748

http://forum.dbpoweramp.com/showthread.php?t=23277

You can improve things by using different source and destination drives (different spindles, not partitions on the same drive), using an SSD, using the DSP effect to limit the number of CPUs, and not using the multi-encoder. I have experience with several programs that offer the multi-CPU encoding, and none of them detect the extreme performance degradation it can cause or adapt to relieve it, so it's just something you have to watch for and deal with yourself.

[jsa307]

Re: Varying encoder speeds?

Interestingly, after trying a few more permutations, it seems the big issue is the multi-encoder feature. With one album's worth of FLAC files set to multi-encode to ALAC and MP3, encoding speeds were around 4x and took about 12 mins. Batch converting the FLAC files only to ALAC was around 300x (10 sec), and batch converting only to MP3 was around 150x (15 sec) -- a tremendous difference!. Toggling the multi-cpu feature and designating a separate drive for input and output didn't make that much of a practical difference because the encoding was so fast anyway. Am I really supposed to see such a large hit in encoding performance when the multi-encoder is used? In my limited attempts to use the multi-encoder with CD ripper (as opposed to files from the hard drive, as in this case), the performance didn't seem nearly this bad ...

[timw0932]

Re: Varying encoder speeds?

Interestingly, after trying a few more permutations, it seems the big issue is the multi-encoder feature. With one album's worth of FLAC files set to multi-encode to ALAC and MP3, encoding speeds were around 4x and took about 12 mins. Batch converting the FLAC files only to ALAC was around 300x (10 sec), and batch converting only to MP3 was around 150x (15 sec) -- a tremendous difference!. Toggling the multi-cpu feature and designating a separate drive for input and output didn't make that much of a practical difference because the encoding was so fast anyway. Am I really supposed to see such a large hit in encoding performance when the multi-encoder is used? In my limited attempts to use the multi-encoder with CD ripper (as opposed to files from the hard drive, as in this case), the performance didn't seem nearly this bad ...

"One album's worth of files" isn't enough to perform a valid test. Try converting a few gigabytes of FLAC files to ALAC or vice versa. If the source and destination drives are the same conventional hard drive, and you are using (say) 4 CPUs, you will eventually enter the random I/O access pattern, such that CPU usage becomes minimal, disk usage hovers around 100%, and throughput plummets. It is inevitable when you have multiple processes all trying to read and write massive amounts of data to the same hard drive over extended periods of time. If you don't observe it, you're not converting enough data, and the amount of memory you have and recent prior access of the files can alter the equation due to caching.

Thus, if you want to test this, I would suggest using a set of files at least as large as the amount of physical memory you have installed in your computer, converting the exact same set of files in the same order each run. This should eliminate the effects of caching between runs and guarantee that you enter the random I/O access pattern. In my testing under these conditions, 2 CPUs is basically a wash compared to 1, and 3 or more hurt things really badly.

As for multi-encoder, I've never used it, and I have no idea if it makes some kind of unique contribution to what you observed. However, if it creates its own threads to compete with the ALAC<->FLAC threads, it would be expected to hasten the arrival to the random I/O access pattern, and the additional amount of disk activity could overwhelm the caching I suspect was skewing your "one album's worth of FLAC files" results.

[Spoon]

Re: Varying encoder speeds?

See the beta section of this forum, there is a new Multi-encoder