title
Products            Buy            Support Forum            Professional            About            Codec Central
 
Page 1 of 2 12 LastLast
Results 1 to 15 of 17

Thread: Need Suggestions on Music Data Tag Reader

  1. #1
    dBpoweramp Enthusiast
    Join Date
    Jun 2020
    Posts
    57

    Need Suggestions on Music Data Tag Reader

    I completed ripping my music CD collection to FLAC with dBpoweramp and creating converted m4a versions of those FLAC files roughly six months ago. I have created two primary and two backup music collection folders on both of my computer's internal and external hard drives. I intend to make it a habit to check those music folders over time such as once every 6 months or year to make sure my song files are good and not somehow damaged, corrupted, or screwed up. I did my initial checks using Mp3Tag to make sure some general song properties such as format and frequency were correct. I also used Ultrasearch app to check to make sure my song files in my assigned folders were not somehow accidentally modified or changed after a certain date. Ultrasearch lets me check files from multiple folders at once but it is not a music app. MP3Tag only lets me load and check one folder at a time. I was wondering if anyone here knows of a music file tag program that lets me check multiple folders at once and can tell me if any of my music files from my ripped CDs have lost or are missing data tag information such as album, artist, genre, etc., so that I can identify and fix those tags if they are missing. I would like to keep my music collection, both source and backup, properly checked and maintained to minimize the likelihood of problems arising due to missing music data and information in cases such as moving my music collection and playlists to a new computer, reloading my music on an existing computer after a factory reset, or transferring the songs to my MP3 player. Does anyone know of a reliable and trustworthy music or music-related program capable of doing the tasks that I just mentioned?

    Thanks.

  2. #2
    dBpoweramp Guru
    Join Date
    Apr 2021
    Posts
    441

    Re: Need Suggestions on Music Data Tag Reader

    Firstly, I use foobar for my tagging QA and WSL for any filesystem checks I make (because I'm more comfortable scripting in Linux than I am Windows).
    Once foobar is setup it's as "easy" as any other tagger, and the fact that it internally stores your tags in a database makes it very quick and powerful to use - and it enables you to run sql against your files to check and update your tags!
    I use foobar for tagging, and have a very functional layout for it making it great for tagging and verification. If you'd like a hand to set it up as I have, let me know as it's pretty straight-forward (I can probably even send you the layout definition for you to load).

    In this image you can see some of the tagging QA checks I have, and my foobar layout. foobar.jpg

    The FLAC format stores the MD5 hash of the decoded audio internally for integrity checking, so the easiest way to check that is:
    Code:
    flac -wt 01.01.flac

    If an error is encountered you'll see output like:
    Code:
    01.01.flac: *** Got error code 2:FLAC__STREAM_DECODER_ERROR_STATUS_FRAME_CRC_MISMATCH
    
    
    01.01.flac: ERROR while decoding data
                state = FLAC__STREAM_DECODER_ABORTED
    To modify your file so that the check should fail you can run something like this against one of your files (that you have copied somewhere)
    Code:
    printf "x01" | dd of="01.01.flac" bs=1 seek=2408448 count=1 conv=notrunc
    If you're using foobar there's a component called foo_verifier that can perform this type of checking for you.


    If your lossless files are CD rips you can go one step further and check against AccurateRip or CTDB with PerfectTunes or CUETools.

    With regard the lossy files I think the only thing you can do (from an integrity point of view) is hash them once you've created them (and after every modification), store it in a file and compare the hashes every 6 months or so. I wrote something to do that for my library, but I'm sure there are other tools out there that do a similar job.


    This is the basic tagging QA check I use in foobar, but I have others including those that check for missing replaygain and that the files are stored in the correct location.

    Code:
    ALBUMTYPE MISSING OR ALBUM MISSING OR ALBUM ARTIST MISSING OR (ALBUMARTISTSORT MISSING AND ALBUMARTISTSORTORDER MISSING) OR ARTIST MISSING OR (ARTISTSORT MISSING AND ARTISTSORTORDER MISSING) OR DISCNUMBER MISSING OR TOTALDISCS MISSING OR GENRE MISSING OR DATE MISSING OR ORIGINALDATE MISSING OR TRACKNUMBER MISSING OR TITLE MISSING OR ALBUMUNIQID MISSING OR DATEADDED MISSING OR "$strcmp($meta_num(artist),$max($meta_num(artistsort),$meta_num(artistsortorder)))" MISSING
    It checks for missing tags and that the number of Artist tags matches the number of Artist Sort tags.

    I also run QA checks on the filesystem for things like:
    Each Disc folder has folder.jpg
    Each Album folder has albumartist.jpg and back.jpg
    CUE sheets only contain lines for FILE|TRACK|INDEX|PREGAP|REM DISCID

    And it goes on :-)

  3. #3
    dBpoweramp Guru
    Join Date
    Nov 2007
    Location
    Florida, USA
    Posts
    5,735

    Re: Need Suggestions on Music Data Tag Reader

    mp3tag will let you load up your entire library at once. I can load my entire library of about 9,000 albums (all in separate Artist/Album folders) at one time in mp3tag. For checking my files to make sure nothing has become corrupted, I occasionally use dbpoweramp to run a BATCH CONVERSION job, "converting" to [TEST CONVERSION]. For my FLAC files, this decodes each file, compares the MD5 hash to the one embedded in the file when it was created, and reports any errors, if they exist. For lossy files, the conversion to TEST CONVERSION will also tell me if the file is corrupted, but there is no checking against an embedded MD5 hash (as this doesn't exist normally within the lossy files).

    edit: I also sometimes use foobar2000 > utilities > verify integrity (as noted, this can also do a batch job to make sure files are not corrupted)
    Last edited by garym; 07-01-2022 at 09:37 AM.

  4. #4
    dBpoweramp Enthusiast
    Join Date
    Jun 2020
    Posts
    57

    Re: Need Suggestions on Music Data Tag Reader

    Thanks to both of you for responding. I am no expert on data tags except understanding the general purpose of them being to identify a song file's properties such as album, artist, and track number. I am also not an expert on all of the audio analysis and verification methods and techniques.

    Garym, with regard to mp3tag, it is a good system, but I would like to check my all four of my source and backup FLAC_Music folders in their assigned locations at once instead of having to do each folder individually. My questions on the Batch Conversion's Test conversion program are does it just analyze the audio properties such as sound quality and proper playback or does it also check tag information like album name, artist, genre, year, track number, etc. and report any missing tag information? Are you saying it can check both audio properties and tag information for lossless files such as FLAC but can only check audio properties for lossy files such as mp3, m4a, and aac? I looked up MD5 hash when reading your response, but I am still not certain that I fully understand its purpose. Also, when doing the test conversion, do I need to setup and assign a destination folder for the test process such as a test folder if Batch Converter has to send the test conversions somewhere?

    Simbun, I appreciate your offer to help. I am considering getting the foobar2000 program. I am not an expert on all audio properties and data tags. I did add ReplayGain detection coding on mp3tag once, but I am still not very familiar with adding new detection codes to a music analyzer or read program. My overall music collection formats are in FLAC, m4a, and mp3 with most of the songs containing ReplayGain. I would appreciate it if you could provide the coding for FLAC, m4a, mp3, aac, and ReplayGain (if applicable, Track ReplayGain and Album ReplayGain properties as well). I would also appreciate it, if it is applicable, that you provide the coding for the data tags that dBpoweramp uses for audio file information such as those from ripped CDs (artist, album, genre, track number, year, etc.) as well as any other data tags you feel are necessary or appropriate. If possible, I would like to be able to put all of the codes in one file or other appropriate single location that foobar2000 would be able to use to perform the check functions as well as be able to create a backup copy of those codes. I would also appreciate it if you can tell me how and where I apply those codes to foobar2000 and then how to use the application to perform the check functions on both single and multiple folders so that I am able to detect whether or not my song files are missing any important album information or other appropriate data.

    Thanks again for your responses.

  5. #5
    dBpoweramp Guru
    Join Date
    Nov 2007
    Location
    Florida, USA
    Posts
    5,735

    Re: Need Suggestions on Music Data Tag Reader

    Quote Originally Posted by Bespin1138 View Post
    Thanks to both of you for responding. I am no expert on data tags except understanding the general purpose of them being to identify a song file's properties such as album, artist, and track number. I am also not an expert on all of the audio analysis and verification methods and techniques.

    Garym, with regard to mp3tag, it is a good system, but I would like to check my all four of my source and backup FLAC_Music folders in their assigned locations at once instead of having to do each folder individually. My questions on the Batch Conversion's Test conversion program are does it just analyze the audio properties such as sound quality and proper playback or does it also check tag information like album name, artist, genre, year, track number, etc. and report any missing tag information? Are you saying it can check both audio properties and tag information for lossless files such as FLAC but can only check audio properties for lossy files such as mp3, m4a, and aac? I looked up MD5 hash when reading your response, but I am still not certain that I fully understand its purpose. Also, when doing the test conversion, do I need to setup and assign a destination folder for the test process such as a test folder if Batch Converter has to send the test conversions somewhere?

    No, the TEST CONVERSION or VERIFY INTEGRITY has nothing to do with confirming "sound quality" or whether any tags are missing. These functions simply determine that the digital file is not corrupt (that is, it can be opened, decoded, and played). In the case of FLAC files, the test conversion is even better as it is reading the "bits" of the current file, calculating an MD5 hash for such audio data and then comparing this to the MD5 hash that was calculated at the time the file was originally created to insure that the audio portion of your files is exactly bit perfect as compared to that audio data when the file was created. None of this has to do with tags such as artist, album, etc.

    edit: and, no, you don't have to create a destination folder for "test conversion". This is all done in memory. No files are written anywhere.

  6. #6
    dBpoweramp Guru
    Join Date
    Apr 2021
    Posts
    441

    Re: Need Suggestions on Music Data Tag Reader

    Quote Originally Posted by Bespin1138 View Post
    My overall music collection formats are in FLAC, m4a, and mp3 with most of the songs containing ReplayGain. I would appreciate it if you could provide the coding for FLAC, m4a, mp3, aac, and ReplayGain (if applicable, Track ReplayGain and Album ReplayGain properties as well).
    When you add (rather than apply which alters the actual audio of a file) replaygain to a file, all you're doing is adding tags that tells the player how much to increase/decrease the files volume to meet an agreed upon loudness standard.
    Even though the different audio formats (such as FLAC, mp3, aac) have different tagging standards (FLAC uses Vorbis and MP3 uses ID3), tagging programs such as foobar hide this complexity by unifying them and thus presenting a consolidated list of tags such as Artist, Album e.t.c.
    So, in foobar if you want to view the replaygain tags you'd use:
    %REPLAYGAIN_TRACK_GAIN%
    %REPLAYGAIN_TRACK_PEAK%
    %REPLAYGAIN_ALBUM_GAIN%
    %REPLAYGAIN_ALBUM_PEAK%


    Quote Originally Posted by Bespin1138 View Post
    I would also appreciate it, if it is applicable, that you provide the coding for the data tags that dBpoweramp uses for audio file information such as those from ripped CDs (artist, album, genre, track number, year, etc.) as well as any other data tags you feel are necessary or appropriate.
    I'm not really sure what you're after here. If you select all your files in foobar, right click and select Properties it will list all the tags for you.

    Quote Originally Posted by Bespin1138 View Post
    If possible, I would like to be able to put all of the codes in one file or other appropriate single location that foobar2000 would be able to use to perform the check functions as well as be able to create a backup copy of those codes.
    Are you talking about the tagging checks e.g. ALBUM MISSING OR ALBUM ARTIST MISSING OR ARTIST MISSING OR DISCNUMBER MISSING OR TOTALDISCS MISSING OR GENRE MISSING OR DATE MISSING OR TRACKNUMBER MISSING OR TITLE MISSING
    Where you put them depends on how you configure foobar (see link at the end of the next answer for where I'd put them as I use the component called facets).

    Quote Originally Posted by Bespin1138 View Post
    I would also appreciate it if you can tell me how and where I apply those codes to foobar2000 and then how to use the application to perform the check functions on both single and multiple folders so that I am able to detect whether or not my song files are missing any important album information or other appropriate data.
    In foobar you can add multiple locations to the "media library" and so by default any checks will be made across the entire library, but it's trivial to filter to individual folders as you can see from the third reply here.

  7. #7
    dBpoweramp Guru
    Join Date
    Nov 2007
    Location
    Florida, USA
    Posts
    5,735

    Re: Need Suggestions on Music Data Tag Reader

    Maybe too simple for what you want to do, but if I was trying to find tracks in my library that were missing metadata (tag fields), I'd simply open my 120,000 tracks into mp3tag. Then, I'd click on the column subheadings (which exist for all my tag fields, like artist, genre, replay gain, etc.), then that will sort that column from highest to lowest) for the contents. If any are missing they will show up all together at either the beginning or end, depending on which direction you are sorting. Then you can fix those. There are fancier ways to do this with filters, etc., but sometimes the simple way is all one needs.

  8. #8
    dBpoweramp Enthusiast
    Join Date
    Jun 2020
    Posts
    57

    Re: Need Suggestions on Music Data Tag Reader

    Thanks to both of you for the replies. I will just continue using the mp3tag and UltraSearch App during my checks, but add in the Batch Converter Test Conversion. My final question is this. My UltraSearch App lists both the Date Created and Date Modified for files listed in its search results. If somehow any of my audio files tags or properties got changed, would the Date Modified be updated to the date those changes took place? I use UltraSearch to make sure my audio files are not modified on or after a certain date such as the date(s) that I am doing the checks. Thanks again.

  9. #9
    dBpoweramp Guru
    Join Date
    Nov 2007
    Location
    Florida, USA
    Posts
    5,735

    Re: Need Suggestions on Music Data Tag Reader

    Quote Originally Posted by Bespin1138 View Post
    My UltraSearch App lists both the Date Created and Date Modified for files listed in its search results. If somehow any of my audio files tags or properties got changed, would the Date Modified be updated to the date those changes took place? I use UltraSearch to make sure my audio files are not modified on or after a certain date such as the date(s) that I am doing the checks. Thanks again.
    hard to say. The file would be re-written, but the question is whether the program re-writing the file changes the file modification date. Many do, some do not. mp3tag has a setting (to tell it whether you want it to change the file modification date).

    Question, what is the basis of your concern that somehow tags get changed without your knowledge. I've never seen this happen in 15 years. A file can be corrupted, but that's different from tags (and easily tested with [TEST CONVERSION]. Do you have roommates, children, spouse, etc. that have access to your music files that are prone to go in and edit them?

  10. #10
    dBpoweramp Enthusiast
    Join Date
    Jun 2020
    Posts
    57

    Re: Need Suggestions on Music Data Tag Reader

    No. I just want to make sure to the best of my ability that my music collection is properly preserved and maintained. It is possible that in the future that my CDs could either badly degrade or could get lost, damaged, or destroyed in cases such as moving, a fire, or a natural disaster. It is also possible that I could accidentally screw something up with my music collection despite my best efforts to be careful. I want to make sure as best as I can, particularly with my FLAC albums that were ripped from my CDs, that they are preserved and play properly well into the future. In addition to backups, I may also store a copy or two of my entire music collection in an external hard drive and store them in a local bank vault and update it once a month depending on how often I purchase or download new music. I don't use cloud storage because I have become more privacy conscious. Plus I am currently job hunting and it is possible I will have to move to a new location once I get a job offer.

  11. #11
    dBpoweramp Guru
    Join Date
    Nov 2007
    Location
    Florida, USA
    Posts
    5,735

    Re: Need Suggestions on Music Data Tag Reader

    In my opinion, the only concern is proper backups (which you seem to be doing) and making sure files are not corrupted (which is easy to do). If tags get edited inadvertently, that's no big deal, as that can be fixed (and is highly unlikely to happen).

  12. #12
    dBpoweramp Enthusiast
    Join Date
    Jun 2020
    Posts
    57

    Re: Need Suggestions on Music Data Tag Reader

    Thanks for the input and advice. If I have any further questions or concerns, I will let you guys know.

  13. #13
    dBpoweramp Guru
    Join Date
    Apr 2021
    Posts
    441

    Re: Need Suggestions on Music Data Tag Reader

    It doesn't look like either of your checks would notice if huge swathes of your files had been deleted, and the checks that you are using are really quite clumsy and incomplete e.g. they only check your music files, not your CUE sheets or images or ...., and they can't know if your tags have become corrupted.

    As I said earlier the proper way to do this is to hash your files and compare those hashes over time.

    @Bespin1138. A simple explanation of hashing can be found here.

    I've written something in bash but I can't seem to find anything equivalent for Windows that would automate it all for you, but he's some code as an example.

    Code:
    @echo off
    
    set MUSIC_ROOT="d:\music_test\"
    
    for /R %MUSIC_ROOT% %%f in (*.*) do (
    
      :: If filesize is 0 certfile will error so output a zero hash
      if %%~zf EQU 0 (
        echo %%f ^| %%~tf ^| 00000000000000000000000000000000
      ) else (
        echo | set/p="%%f | %%~tf | "
        certutil -hashfile "%%f" MD5 | findstr /V ":"
      )
    )
    The above would output the data:
    Code:
    FILENAME | MODIFICATION DATE | HASH VALUE
    D:\MUSIC_TEST\2010 - 1962-1966 (The Red Album)\album.info | 07/06/2020 13:42 | b801ade7cf1873f55e6c7dc09f06abf4
    D:\MUSIC_TEST\2010 - 1962-1966 (The Red Album)\CD01\01.01.flac | 16/02/2022 16:51 | 1ecacec72f57c09b6dec40e6ffb8acdb
    D:\MUSIC_TEST\2010 - 1962-1966 (The Red Album)\CD01\01.02.flac | 16/02/2022 16:51 | a6868a2056ea80fdcdfe67df294fbceb
    D:\MUSIC_TEST\2010 - 1962-1966 (The Red Album)\CD01\01.03.flac | 16/02/2022 16:51 | 23faee6b5569a74ffa0c04c1e67284af
    D:\MUSIC_TEST\2010 - 1962-1966 (The Red Album)\CD01\folder.jpg | 17/04/2021 19:48 | f4aadf35acc6316a43c42c0152e563e3
    D:\MUSIC_TEST\2010 - 1962-1966 (The Red Album)\CD02\02.01.flac | 16/02/2022 16:51 | af72fc044095c0f8394c9b3c7c689422
    D:\MUSIC_TEST\2010 - 1962-1966 (The Red Album)\CD02\02.02.flac | 16/02/2022 16:51 | 4ad37da0f5fbfb142ddce0428e78aa7b
    D:\MUSIC_TEST\2010 - 1962-1966 (The Red Album)\CD02\02.03.flac | 16/02/2022 16:51 | 1a79b6231624381f2a2e38bb9a1c7a19
    The idea is that if you ran the process initially, and then X months down the line and diff'd the two files you'd be able to tell what files are New, Deleted or Modified (and if the modification date hasn't changed then it's been corrupted).

    It's actually a little more involved than that as you'd want to add new files/modifications to the list as you go, but hopefully you get my point.

  14. #14
    dBpoweramp Enthusiast
    Join Date
    Jun 2020
    Posts
    57

    Re: Need Suggestions on Music Data Tag Reader

    Thanks again all for your input. For safety, before the Test Conversion, since the Test Conversion is new to me and I never knew that it existed until it was mentioned in this forum, I created a test copy of one of my FLAC music folders and and took note of the number of files and folders within that test folder with Windows Properties Tab, used Ultrasearch to note the audio file with the latest date modified date within the test folder, and performed the test conversion. After the test conversion, I looked up the folder and file counts and latest date modifed dates again within the mentioned programs and as far as I could tell the folder and file counts and date modified dates looked to be the same as before the test conversion. But before I use the test conversion program on my main music folders, I have one final question, based on the information that I received both in this forum and on one other dBpoweramp forum that I could find covering the test conversion program, am I correct in understanding that the Test Conversion program within dBpoweramp's Batch Converter program only inspects the audio files making sure they can be opened, read, and played, reports any audio files that either are unplayable or have other playback issues, does not change or delete the files in anyway, and does not make copies of the audio files or transfer or copy the audio files to any other location(s)?

    Thanks again.
    Last edited by Bespin1138; 07-05-2022 at 01:12 PM.

  15. #15
    dBpoweramp Guru
    Join Date
    Nov 2007
    Location
    Florida, USA
    Posts
    5,735

    Re: Need Suggestions on Music Data Tag Reader

    Quote Originally Posted by Bespin1138 View Post
    ;, read, and played, reports any audio files that either are unplayable or have other playback issues, does not change or delete the files in anyway, and does not make copies of the audio files or transfer or copy the audio files to any other location(s)?

    Thanks again.
    yes you are correct. Test conversion works as you indicate. I have run it many times on my 120,000 track collection with no problems whatsoever.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •