Adding large folder to batch convert = PROBLEM!!!

[yigalnavon]

Hello
I am trying to convert 140Gb folder with 140,000 WAV files to FLAC .
The Wav files are music studio loops .
When i choose the folder in "Batch convertor" and press the convert button...
My mouse pointer change to blue circle and nothing append.
Then i open "task manager" see "dbpoweramp batch converter" not responding,
And see the process "music converter" using 25% cpu and memory usage get bigger and bigger...
But very slowly , After 15 minute it get only to 150Mb... and it continue on getting bigger but very slowly !
After 40 minute the "music convertor" window open! ALELUYA :-)
I have test it with small folder with 20,000 files and it worked sort-of fine (opened after 1 min+- )!
Hope you can help me on this one... (been using foobar2000 for conversions but needs more options power)
Thank you in advance.
OS:windows 8.1 CPU:q6600 MEM:8Gb

[mville]

Re: Adding large folder to batch convert = PROBLEM!!!

What help are you asking for?

It seems to be working, just that it takes a long time when you select 140,000 files. Or am I not understanding you correctly?

[BrodyBoy]

Re: Adding large folder to batch convert = PROBLEM!!!

I have test it with small folder with 20,000 files and it worked sort-of fine (opened after 1 min+- )!
Hope you can help me on this one... (been using foobar2000 for conversions but needs more options power)
Thank you in advance.
OS:windows 8.1 CPU:q6600 MEM:8Gb

There's really no help needed...as you already discovered, dBp works fine when you convert manageable batches. 140,000 files is not a manageable batch on your machine.

Try running a 10,000 file batch. If your computer isn't too bogged down by that, do the rest of the files in similarly-sized batches.

[yigalnavon]

Re: Adding large folder to batch convert = PROBLEM!!!

So...this is normal !?
I wish the 40 min wait was embedded in the conversion process... you know 2015...
(As long as the user don't click "List/Rename", i can't see the problem).
Any way... I looked at the converted flac files and see that in some of them it looses the BPM info
(Originally was acidized WAV) Why some wavs convert with no problem and some looses the BPM info in the process ?
Again thank you for your help in advance.
P.S there where no errors in the conversion process.

[BrodyBoy]

Re: Adding large folder to batch convert = PROBLEM!!!

So...this is normal !?
I wish the 40 min wait was embedded in the conversion process... you know 2015...
(As long as the user don't click "List/Rename", i can't see the problem).

Not sure what you mean, "embedded in the conversion process....2015?" The amount of time it takes to read all the files is a function of how large the batches are. You are running very large batches. Just run smaller batches....a few thousand at a time, perhaps....if you don't like the long initial read times.

Any way... I looked at the converted flac files and see that in some of them it looses the BPM info
(Originally was acidized WAV) Why some wavs convert with no problem and some looses the BPM info in the process ?

That's a metadata issue. Acid adds its own "proprietary" data that isn't well-supported in other programs, so I suspect it has something to do with that. Take a look at the BPM data in some of your original files....perhaps the files that converted with BPM data have a standard BPM or TBPM tag, while the those that didn't have just Acid data that dBp doesn't use? (I've never used Acid files, so I'm not familiar with which data dBp supports or recognizes.)

P.S there where no errors in the conversion process.

Glad to hear it.

[yigalnavon]

Re: Adding large folder to batch convert = PROBLEM!!!

Hi BrodyBoy
Thank you for your kind answer .
Regarding the "long initial read times" it would be very nice if we had a progress bar.
Regarding the Sony acid bpm data , i couldn't able to find a way to fix it.... :-(
10x again !

[nab128]

Re: Adding large folder to batch convert = PROBLEM!!!

I agree this is a problem. This program is designed for professionals so how can you say 140,000 is too much? I think you will see this delay no matter how fast your PC is. I have i7 with 12GB RAM and it is still very slow to start. Then you have to wait another long time for "Examining Source and Overwrite Check" even if you are converting to an empty folder.

Why can't we choose a large folder to convert, click ok and walk away and come back in the morning with the job finished. I don't want to wait 40 minutes to click another button! Can you see my point?

This is what yigalnavon is talking about and I hope Mr. Spoon can make a "one click" convert option for us! Or at least explain why it is not possible.

[Dat Ei]

Re: Adding large folder to batch convert = PROBLEM!!!

My impression is that the amount of time increases disproportionately to the number of files. Being a developer myself, I've seen such behaviour very often, when I build up sorted lists and sorted the lists after every insert instead of building up an unsorted list first and then to do the final sort once.

I only manage about 27.000 files but even on my system (i7-3930K with 64 GB RAM) I suffer from the delay. In the same time, I stitch and produce panoramic photographies with 300-400 pics as input, which is an even higher load and complexity than the preparation of a list fpr the batch convert. Just my speculation...


Dat Ei

[Spoon]

Re: Adding large folder to batch convert = PROBLEM!!!

We will take a look at the design for R16

[Shootkin]

Re: Adding large folder to batch convert = PROBLEM!!!

The problem still persists in 2020. I downloaded dMC-R16.6-Ref-Trial and tried to batch convert a folder (3GB, about 4000 wav files). Listing and/or selecting files takes per 16 minutes each operation. Closing the program takes another 16 minutes or so, I just killed it in task manager. Conversion itself is lightning fast on all 12 cores.

My hardware is: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz (6 cores), 32GB RAM, 256GB NVMe system disk, 1 TB SSD secondary disk (yes, I unchecked the "Pre-cache tracks..." in Advanced Settings). My Windows 10 is Home x64 (1909 build 18363).

In the task manager the DMCFileSelector is "Not responding", takes 16% CPU and 106 MB RAM. In the CPU activity utilization graph by logical units I see such a picture: CPU0 10%, CPU2 100%, CPU4 40%, CPU6 20%, the rest are idle.

Dear Mr. Spoon, please fix this because I plan in the next couple of months to convert hundreds of thousand files from my realy huge sample library. Thank you!

[Spoon]

Re: Adding large folder to batch convert = PROBLEM!!!

Try R17 which is in the testing section of this forum, it had much work done in this area.

[Shootkin]

Re: Adding large folder to batch convert = PROBLEM!!!

Unfortunately, nothing has changed. Just installed the latest R17 but the behaviour is the same. Batch Converter lists my 10K files, shows "Reading Meta Data.." in the status bar and that's it. I can not select them in the list because the app is not responding. I even turned on and off the Windows Defender but it seems unrelated.

[Spoon]

Re: Adding large folder to batch convert = PROBLEM!!!

You should be able to Open Batch Converter, select your root folder and click convert which opens the encoding options, no metadata is read until the conversion actually takes place.

[Shootkin]

Re: Adding large folder to batch convert = PROBLEM!!!

Greetings! Now the things went much better!

But it's kinda workaround anyway. I still unable to select directly the root folder without needing to wait an hour or so before the job start. But if I put my audio folder into any empty one and select that upper directory in the Converter's list — everything works just fine. Here's a small screen capture of what I did. May be you find some more elegant way to make it happen.

Mr. Spoon, thank you very much for helping us

[Spoon]

Re: Adding large folder to batch convert = PROBLEM!!!

I would not select the folder wav, instead expand the one above then click the [ ] box next to wav, it will be fast then.