Most Expedient Processing Method for a Large >250k Collection?

[Mrmb]

I have over 250,000 (16-24 bit + DSD) FLAC tracks. An internal 16tb HD (Hard Drive) presently holds the ~250k bulk of these.

There are 2 other internal HD's also with tracks. One HD has 1-folder and the other HD has 2-folders.

I started a De-Dup process using the ~250k HD and added 1 of the 3 other folders. Obviously, this is taking days to do.

When finished, will the "listening" done from these HD's/Folder be somehow saved? I'm asking because when I include the other 2-folders/HD, will this "listening" process begin anew? If that's a given, so be it. But can I do something to save the already "listened" to files/data? Something that would preclude having to start over again and thus, hasten the additional 2-folder "listening" process?

It obviously would have been more prudent to have asked the: what is the most expeditious method of handling my collection question before beginning.
Nevertheless, essentially, that is the question.

BTW, I have no qualms with halting the De-Duping that is currently ongoing (approx. 80k tracks have been processed) and begin anew, if that is a recommendation. Thus &/or, perhaps should I have begun with the Album Art or ID Tags and then ran De-Dup?
NOTE: My metadata and album art is fairly accurate given my collection's size. But I'm sure a "clean-up" would NOT be wasted. However, because over the years, everything was ripped using dBpoweramp and accurate rip, the rip accuracy isn't a concern.

How would you recommend I process my tracks -- I.e. the quickest method?


[SIZE=1]DIY PC:
Windows 10
Board: ASUSTeK COMPUTER INC. P8Z77-V PRO Rev 1.xx
3.50 gigahertz Intel Core i7-3770K

Multi-core (4 total)
Hyper-threaded (8 total)

16,326 Megabytes Usable Installed Memory[/SIZE]

[Spoon]

Re: Most Expedient Processing Method for a Large >250k Collection?

Yes the listening is saved and only has to be done once, with such a large music collection it is going to take quite a while to process.

[Mrmb]

Re: Most Expedient Processing Method for a Large >250k Collection?

Thanks Spoon,

To clarify and help me understand what's going on, can I assume by "saved", the "listening" is saved in a file, in perhaps the dBpoweramp folder? That is, there is nothing I do to save it, or again call it up, after I log out of PerfectTunes?

Thus, if I log out of PerfectTunes and then re-activate it and specify/select the folders already "listened" to, their information will be automatically on-hand (without any user intervention)? I can then add a folder(s) NOT yet listened to along with the folder(s) listened to and time will be spent "listening" to the new folder(s), but NOT to the old, already listened to folder(s)? Does this logic only apply to the De-dup process, or will it equally apply to the other 3-PerfectTunes functions -- I.e. ID Tags, Album Art & Accurate Rip? For example, after starting-up ID Tags, when I browse and select the folder(s) already listened to (in De-Dup) will their data automatically be available to the ID Tag program?

Can the already listened-to database be exported to another folder, another PC? If so, how would that be done -- the file identified etc.? Can I presume that IF the Folder(s) listened to remain unchanged, all will be well? What happens if adds or deletes are made to the folder(s) listened to? Only those changes will be listened to, or...?

As you can glean, my questions and concerns revolve around maintaining/saving the file/data and NOT overtly, or accidentally loosing, or deleting it. Especially given the many days it took to compile it. So, I'm trying to understand how the process works and is saved, &/or can be subsequently used again?

[Spoon]

Re: Most Expedient Processing Method for a Large >250k Collection?

You should not remove any scanned folders as that might retrigger the relistening. PerfectTUNES has its own internal database.

You can find the database in

%appdata%\PerfectTUNES

[Mrmb]

Re: Most Expedient Processing Method for a Large >250k Collection?

You should not remove any scanned folders as that might retrigger the relistening. PerfectTUNES has its own internal database.

You can find the database in

%appdata%\PerfectTUNES

After several days, the De-dup process finished. I exported a copy of the output to Excel and then began analyzing the 40,000+ duplicates. I had done this several years ago, so I had to refresh my previous experience. It was going well. After getting a handle on the output, I deleted a few tracks; when as Murphy's Law would have it, some distant thunder was heard and you guessed it: a 1-2 second power outage occurred. So, POOF, there went the PC & the De-dup output, accompanied by a couple expletives from me!

I restarted the PC and restarted PerfectTunes loading the folders De-dup had already "listened" to. The previous output/database seems to be rebuilding, albeit slowly. It will be interesting to see how long the "Detecting Duplicate" process will take. I was hoping for a more instantaneous process. But an hour or a few, is far better than days Moreover, trying to use some sort of music player or tagging application to grossly detect duplicates would be a nightmare, compared to De-dup. So, I'm very happy to have such a powerful program as PerfectTunes