How does TuneFusion know what files in a previously sync'd folder need a rewrite

[vilsen]

Re: How does TuneFusion know what files in a previously sync'd folder need a rewrite

IMO, if unsure, it is smarter to confirm rather than assume, as demonstrated perfectly here by schmidj.

Yes, I like the confirm approach too. Since schmidj asked for a clarification and also did a feature request for a different wording than the well used "preserve" expression, there was room for some background. IMHO "preserve" works ok and it would only confuse things to change it.

But this is merely a side note to the real issue here, which could be a bug that schmidj is on to.

[mville]

Re: How does TuneFusion know what files in a previously sync'd folder need a rewrite

schmidj, it looks like the sync internal database is screwed. When I experienced issues in the past, I simply removed the sync and started again. Now, I'm running R1.7 and currently have 3 syncs which, touch wood, are behaving themselves.

I haven't tested all the various options in TuneFUSION, but I suspect there are still issues lurking within, as seems to be the case here.

[schmidj]

Re: How does TuneFusion know what files in a previously sync'd folder need a rewrite

It appears that somehow in my experimenting that the permissions for the files in the folder that has the issue got changed such that windows explorer can see them (and move them around on the (Linux)server but TuneFusion can't see them. I can see that in looking at the file properties in windows explorer now. I need to sort out whether that is an issue on the server (possibly with Samba) or a Windows issue. So, I need to straighten that out and will then try my experiment with changing tags using mp3tag and then resyncing using TuneFusion again. This may not be a TuneFusion issue??? (In fact I'm a little suspicious that mp3tag may have locked the files (the whole folder including the files where I modified the tags) (by changing the permissions) when I was editing or looking at them and never "unlocked" them on the Windows side. I'll report back after further investigation.

Thanks

John

[GBrown]

Re: How does TuneFusion know what files in a previously sync'd folder need a rewrite

(In fact I'm a little suspicious that mp3tag may have locked the files (the whole folder including the files where I modified the tags) (by changing the permissions) when I was editing or looking at them and never "unlocked" them on the Windows side.

I don't think mp3tag has any way of "locking" a folder or file, or even a specific tag field, unless perhaps you have the file open in mp3tag and are actively saving to it. In fact, there has been some discussion on that forum about requesting a feature to do this, but at this time there is no way to do it.

[schmidj]

Re: How does TuneFusion know what files in a previously sync'd folder need a rewrite

I'm still sorting it out, but the server I purchased a year ago never had SAMBA properly configured. My bad. I'm not sure that was/is the only issue, it will take a while for the correct permissions to propagate through the server before I can experiment more. It is likely that mp3tag had nothing to do with it, but there is still a mystery why some files behaved "correctly", others didn't and some were visible to explorer but not TuneFusion. However I realize that explorer is running through my login while TuneFusion I believe is running as a service, which probably affects how it reacts to the permissions.

[schmidj]

Re: How does TuneFusion know what files in a previously sync'd folder need a rewrite

I do wish Illustrate would develop a version of TuneFusion which would run as an app on my QNAP NAS. I wouldn't have to leave my editing PC running to allow it to run. But, given the issues with QNAP and Access, I can understand why Illustrate hasn't gone in that direction. An alternative would to have a version to run on a dedicated Pi computer or similar and talk via Ethernet/Wifi to the QNAP to sync directories.

[schmidj]

Re: How does TuneFusion know what files in a previously sync'd folder need a rewrite

So here is the latest on my battle with TuneFusion (which generally has behaved well in the past, except for the direct syncing with my Android phone, for which I have a work-around. (more on that later in another post)

I found that my now over a year old NAS server had an issue with SAMBA (the app that allows Linux files to work with Windows and visa-versa) security settings never having been properly configured. I corrected that (which needed doing to properly secure the files), but that didn't solve the issue. (At least)One folder (album) of tracks refused to sync,it was like TuneFusion never saw it. It was the actual audio files (mp3's) since if I moved a file from another folder into the folder that had the "invisible" files, and moved one of the "invisible" files to another folder. the file from the other folder now synced (as the only file in the output in the folder with the files that wouldn't sync and the non-syncable file moved to another folder still didn't sync. Clearing the TuneFusion database didn't help. No matter what I tried, even changing the fiename and some of the tags in the "invisible" files (in windows explorer), nada...

Finally, I set up a new sync that included the "invisible" files and a portion of the 130,000 other files, a different new directory on the same destination drive, otherwise identical settings. Surprise (or maybe not), all the files synchronized, including the "invisible" in the original sync ones. So something is corrupt in the original sync, something that clearing the database or anything else I tried didn't fix. I've now set up the new sync to sync all 130,000 files. Tomorrow about this time, when it is finished, I'll know for sure the results. Then I'll retest the original issue of changed tags not being synced.

John

[schmidj]

Re: How does TuneFusion know what files in a previously sync'd folder need a rewrite

OK, the latest results:

I created a new sync, as I said in the last post. Identical to the old one except the destination was a new directory on the same drive as the old one. It ran for almost 24 hours, completed successfully. 130,537 files synced. Note, I didn't reinstall TuneFusion, just created a new sync, added to the same install.

Then ran the previous sync again, 126,523 files synced. (somehow over 4000 files missed) but it says completed successfully. Note that before running this sync, I had clicked on the clear the database option. A week or so ago I had run it with the resync everything option. But somehow, it never cleared everything and kept missing over 4000 files...

On the new sync, after it completed, I tested changing tags with mp3tag, which does change the datestamp on the files with the config box unticked, and they did update correctly when resynced. Also tried changing tags with the dBpa right click editor. It also updates the file datestamps, (I don't see any option to not change them but there might be a DSP option to not change them.) And again, the tag info was properly updated when I ran the sync again.

So, when you don't have an apparently corrupted sync, it does update tags correctly. But what bothers me is that it was "off the rails" in a significant manner and yet reported that it completed successfully but wasn't syncing some 4000 files, repeatedly.

I'm going to see if I have (or can easily get) software to compare the two directories, to see what was left out. I know one batch of files, but that only accounts for a little over 100 of the 4000 files.

John

[schmidj]

Re: How does TuneFusion know what files in a previously sync'd folder need a rewrite

So, I have most of the answer, a part that makes me feel stupid! I've spent several weeks sorting out sloppy tags, fixing artist's names, properly indicating multiple artists, genres, greatly sorting out my previously messy genre choices, fixing typos and missing tags, etc.

I also updated TuneFusion to the recent latest update.

I ran the old supposedly defective sync. It was still missing (not syncing) some 4000 or so tracks. Ran the new sync I had created, supposedly identical but to a different destination directory. It found all the tracks.

So, I went back and looked at all the details of the supposedly identical syncs. And I now feel the fool. When I looked at the list of genres to include/ignore, the "defective" sync had been set to not sync all the Christmas (and similar) genres! I forgot I had done that several years ago so I wouldn't be listening to Christmas carols in my car during the summer. (I have the app in my phone set to play tracks at random). And that's where the 4000 tracks went.

So, I have to apologize for this wild goose chase.

Now that leads to the following question: In tracks with multiple genres, is the absence of a genre from the sync list inclusive or exclusive? Let us say a track with genres Christmas; Rock. Christmas is not ticked on the genre list, rock is. Will that track be synced? Does the codec make a difference? (FLAC, mp3, m4a-MP4)? Does the order of the genres in the original track metadata make a difference (Rock or Christmas first)?

In my mind there is no "right" answer unless there was a way of indicting whether it was to be inclusive or exclusive. When I use it to avoid the Christmas songs in the summer, I'd like it to leave all the tracks with a genre "Christmas" out of the sync, even if they have another genre like "Rock that is included. But if I'm making a sync of let us say "rock", I'd want the sync to include the tracks that have genres of rock and blues, even if blues was not checked. I want to eat my cake and have it too... Actually, if I knew it was exclusive (i.e. all the genres on a track had to be selected for it to be synced, not just any of the genres), I can deal with it, as I can just make sure that all the required genres are checked in Tune Fusion, but if it is inclusive (it is synced if any of its genres is checked in TuneFusion), then I have no way to for instance leave out all the Christmas songs, at least with the way I have my genres entered.

Spoon, Peter, do either of you have the answer to this?

Thanks for your time, including your time I may have wasted by not checking carefully how I had the sync set up.

[Spoon]

Re: How does TuneFusion know what files in a previously sync'd folder need a rewrite

If you exclude all Genres, then it is based on inclusion, any genres you add are only included.

If all are included, and a few are unchecked, then it is based on exclusion, so a sub genre should pick up on that.

[fphredd]

Re: How does TuneFusion know what files in a previously sync'd folder need a rewrite

I've been using dBpoweramp for years (just paid for updated version, thanks for the discount!) and then purchased TuneFusion to re-convert my library of ~80,000 tracks from FLAC to MP3 (travel versions!).

The full job took a couple of weeks, which was fine, though I did restart my computer a number of times. When I knew I'd be rebooting, I'd often delete a few of the most recent folders (albums) as I wasn't sure how it would handle a partially finished album on a restart. Due to this, and any other unknown factors if they exist, the mp3 folder is missing about 3500 tracks. As a noob, I deleted the sync and figured I'd recreate it, select the option to Detect and Rewrite Missing Files on Destination and it would elegantly fill in the blanks. When the sync starts running, however, it's rewriting the files from the first. Is there no way to avoid this complete rewrite? Figuring user error somewhere? Thanks!

[Spoon]

Re: How does TuneFusion know what files in a previously sync'd folder need a rewrite

It writes a file in the root folder which contains the list of files converted (for removeable media).

[fphredd]

Re: How does TuneFusion know what files in a previously sync'd folder need a rewrite

So if I've deleted that original sync definition the db gets deleted and there's no way around reconverting/overwriting the entire library (networked drive)?

[Spoon]

Re: How does TuneFusion know what files in a previously sync'd folder need a rewrite

Yes

[fphredd]

Re: How does TuneFusion know what files in a previously sync'd folder need a rewrite

Yes

If I move back to Batch Converter, will it scan what's there and correctly report what doesn't need to be redone?