mass removal of duplicate tracks

[ICEBROOD]

Re: mass removal of duplicate tracks

Hey guys, New member here and first want to say Merry Christmas and Happy Holidays to all!

I have read the replies here for the most part and gather that this program scans all directories that we point it to obviously? My question is once the results are shown do we have to go back and hand select each copy we have delete for every unwanted music file? I have over 24k duplicates. I am looking for a program that will scan all directories leaving ONE file (as suggested by the member who initially posted here) in the directory I choose as my music source and highlight ALL other files for easy 'mass' deletion. Is there such a beast because my hunch is this program is asking me to go through each and every result it produces and select each I wish to delete. That would be all fine and good if I did not have copies of music spread out over my C/...J/...K/ drives etc. If any of us needing a program to delete unwanted duplicates had all 'those' duplicates sitting in one or another drive then it would be rather easy to delete them now wouldn't it? Fact is they are every where and maybe I am missing an easier process in this program (which looks pretty awesome on first use) to achieve my goal here. Put simply...does this program leave one copy behind if scanning all directories (that knowingly have music files I have pointed too) or is that up to me to single out the file I want to retain on every result? Many Thanks for any help.

[Spoon-]

Re: mass removal of duplicate tracks

If PerfectTUNES is no longer indexing, then it does not have access to the music, try to browse with Windows explorer to the music, if it asks for a password enter it, then PerfectTUNES will be able to access also.

[mville]

Re: mass removal of duplicate tracks

As far as i can tell the orphaned files are in the folders too.

I expect the tags for these files are incomplete. You will need to compare the tags for all the files for the album and make sure, where applicable, that they are the same.

[Tim Allen]

Re: mass removal of duplicate tracks

Thanks.

I thought the reference to 'fixes' in the instructions would be where album art and duplicates etc had to be inserted through PerfectTunes.

When I allow PerfectTunes to rescan my library, it says there are no albums.... But they are there when i look in my library, and when I play them.

As far as i can tell the orphaned files are in the folders too.

I have tried deleting my library from the PerfectTunes window, and then reinserting it. That seems to be doing something.

[mville]

Re: mass removal of duplicate tracks

Many thanks. I have now managed to run mass removal on just my Diskstation and it worked. I also managed to correct the album art. However, 34 files continued to show as duplicated, and when I trued to delete them, a message said I could not do so, and should check if I have permission. I assumed that was because of some sort of 'fix' I had used.

What fix are you referring to?

So - perhaps unwisely - I clicked 'clear data base cache'. That certainly worked in removing the duplicates, but it also means that no albums at all now appear in PerfectTunes. The stuff is still in my music library. Can you explain what i have done? Is is a problem.

You will need to allow PerfectTunes to re-scan your library and re-build the database.

Also, something that has persisted after running PerfectTunes is that most of my library is in folders and albums. However, quite a few files seem not to be, and just show as individual tracks. Is there an explanation for that? Does it matter? I have done the same thing in all cases - i.e. ripped CD albums as whole albums and inserted album art if it has not immediately shown. so it is not clear to me why all the files are not in folders.

Have you checked whether these orphaned files reside in the correct album folders or perhaps the album tags are incorrect for these files. You will need to check, manually.

[Tim Allen]

Re: mass removal of duplicate tracks

Many thanks. I have now managed to run mass removal on just my Diskstation and it worked. I also managed to correct the album art. However, 34 files continued to show as duplicated, and when I trued to delete them, a message said I could not do so, and should check if I have permission. I assumed that was because of some sort of 'fix' I had used. So - perhaps unwisely - I clicked 'clear data base cache'. That certainly worked in removing the duplicates, but it also means that no albums at all now appear in PerfectTunes. The stuff is still in my music library. Can you explain what i have done? Is is a problem.

Also, something that has persisted after running PerfectTunes is that most of my library is in folders and albums. However, quite a few files seem not to be, and just show as individual tracks. Is there an explanation for that? Does it matter? I have done the same thing in all cases - i.e. ripped CD albums as whole albums and inserted album art if it has not immediately shown. so it is not clear to me why all the files are not in folders.

[Spoon-]

Re: mass removal of duplicate tracks

You should not have your music folder and discstation at the same time, there should be only 1 item listed under library.

[garym]

Re: mass removal of duplicate tracks

Thanks. I could try that, but the second configuration of Diskstation was put in only last night. I did that to see if I started again and left it on overnight it would work better. It did.
However, the duplicate tracks are the same as they were before. As far as i can tell, no more have been added. It is basically just an extra copy of everything.

Can you temporarily unplug the 2nd drive in your NAS (the one that contains the backup*) and then rerun de-dup?

*EDIT: "backup" is not quite correct. I hope you know that RAID is *not* a backup. It is a data redundancy that reduces downtime (useful for banks, shopping websites, etc.). But it certainly is not a backup, so I hope you have a separate backup of your files. Google "why raid is not a backup" if you'd like more info or to read some horror stories of people who thought RAID was useful as a backup.

[Tim Allen]

Re: mass removal of duplicate tracks

Thanks. I could try that, but the second configuration of Diskstation was put in only last night. I did that to see if I started again and left it on overnight it would work better. It did.
However, the duplicate tracks are the same as they were before. As far as i can tell, no more have been added. It is basically just an extra copy of everything.

[mville]

Re: mass removal of duplicate tracks

DISKSTATION/Music is indicated. I left it running overnight, and it seems to to have connected to that library. It has now corrected the album art. I am not sure what the problem was. Maybe it was just very slow - although there are only about 40 cd albums in the library. Possibly, it will help if I plug my computer directly into the NAS drive.

My concern about mass deleting duplicates remains. As far as i can see, everything I have put in my DISKSTATION music library has been duplicated. There are two copies of everything showing. My suspicion is that what I am seeing is one that shows in my library when i open it, and one that is on the second WD Red hard drive, which the NAS drive uses for backing up all files. Does that make sense? If so, having two copies of everything in PerfectTunes is OK. However, it would be better if I can do as suggested - i.e. that I do not include the back up files in PerfectTunes. How do I do that?

DISKSTATION/Music is indicated twice, which might explain why you are seeing duplicates.

Also, you have configured 2 other locations. Only you will know what is contained in these locations, but if you have copies of your files within these locations, this might also explain why you are getting duplicates.

If you are sure that your audio files are in DISKSTATION/Music, I suggest you start again, removing ALL locations (and maybe clear database cache). Then, Add Folder again for DISKSTATION/Music, so this is your only indexed location.

I'm not entirely sure why PerfectTunes allows users to add duplicate locations, but maybe Spoon can comment on this.

[Tim Allen]

Re: mass removal of duplicate tracks



Here is a screenshot as requested. As you see, DISKSTATION/Music is indicated. I left it running overnight, and it seems to to have connected to that library. It has now corrected the album art. I am not sure what the problem was. Maybe it was just very slow - although there are only about 40 cd albums in the library. Possibly, it will help if I plug my computer directly into the NAS drive.

My concern about mass deleting duplicates remains. As far as i can see, everything I have put in my DISKSTATION music library has been duplicated. There are two copies of everything showing. My suspicion is that what I am seeing is one that shows in my library when i open it, and one that is on the second WD Red hard drive, which the NAS drive uses for backing up all files. Does that make sense? If so, having two copies of everything in PerfectTunes is OK. However, it would be better if I can do as suggested - i.e. that I do not include the back up files in PerfectTunes. How do I do that?

[mville]

Re: mass removal of duplicate tracks

I only want one copy of a file in my music collection (which is backed up on my NAS drive). The de-dup program suggests there are are many duplicates. I am not sure why that is.

Probably because you have included both your music collection location and the music collection backup location in PerfectTunes.

Also, I have been assuming that I am accessing my music library - which is in my NAS drive. However, your answer suggests that is not the case. So which 'library' is being read?

It is not possible to answer this question. It is the location(s) that YOU configure in PerfectTunes.

The one in the NAS drive comes up in the window. If is is not actually reading that one, is PerfectTunes automatically only reading the one in my computer? If it is the one in the Synology NAS drive (which appears as DISKSTATION/music), is that affected by the back up of files on my second hard disc (you seem to suggest it is not)?

Spoon is not suggesting anything, only advising you NOT to delete anything until you are absolutely sure you know what you are deleting and how to configure the location that is being indexed in PerfectTunes De-Dup.

Please can you post a screenshot of what you have showing in PerfectTunes Music Library?

[Spoon-]

Re: mass removal of duplicate tracks

For one of the duplicates click the 'i' and look where it says the folder is for that item, then click 'i' on the next item, what folder location is written?

Or if you want to see the location of all duplicates, click the spanner icon top right >> Export Duplicate List

[garym]

Re: mass removal of duplicate tracks

perfect tunes is reading the library that you pointed the software to. You have to enter something. It could be your NAS. As spoon noted earlier, you should be able to click on these files or the i button and see exactly the computer path where the file is located. Could be your NAS, could be your backup, could be your local PC, or ?????.

[Tim Allen]

Re: mass removal of duplicate tracks

Many of your questions, only yourself can answer because it is your own music collection. You should not delete anything until you are certain where the duplicates are from. In the de-dup program, click the spanner icon top right and see which location is being index, it should be only 1 location, no your nas drive. The 'i' info button in dedup will show where the files are located for a detected duplicate.

About album art, you can run it manually, that is for each album you click the 'fix' button next to the album, to just do that album.

Thanks so much. I only want one copy of a file in my music collection (which is backed up on my NAS drive). The de-dup program suggests there are are many duplicates. I am not sure why that is.

Also, I have been assuming that I am accessing my music library - which is in my NAS drive. However, your answer suggests that is not the case. So which 'library' is being read?

The one in the NAS drive comes up in the window. If is is not actually reading that one, is PerfectTunes automatically only reading the one in my computer? If it is the one in the Synology NAS drive (which appears as DISKSTATION/music), is that affected by the back up of files on my second hard disc (you seem to suggest it is not)?

So, if I just click delete all duplicates, what will happen? In each case two files appear in the windows. Will just one be deleted, or both?

On correcting the art work - yes I know I can correct the individual ones that come up. However, not all albums come up when I use PerfectTunes (maybe because it is not actually read my DISKSTATION /music library). Yet, running PerfectTunes has inserted art in those albums that do not come up - sometimes wrongly, and I can't find a way of accessing them from PerfectTunes.

Am I missing something basic? Doing something daft?
Sorry if I am being thick.