Tweet
File Corruption Ripping to SMB-Connected NAS on macOS
Hello All,
For weeks now, I have been trying to track down why I get "Chunk" errors ("Chunk is out of file") when ripping to NAS, but not when I rip to a local temp dir, then move the files to NAS. Well, I now have some preliminary answers--but, it is not clear whether this is a macOS issue, an SMB issue, or a NAS issue, and whether it is only affecting macOS or impacts Windows (and *nix) as well.
My first clear indication that corruption when writing to NAS is an actual problem occurred when I was running a script to check for organization issues in my music library. The script was executed by:
and the script, my music library, and all dirs were on my NAS, as was my macOS current working directory in Terminal. As the script ran, I saw normal results displayed via tee. However, later in the day when I went back to review the vc.log file, all but the last few lines were nulls--a few thousand nulls, about one for every line which should have been in the log.
Searching, I find various reports of similar problems on numerous developer forums. If the forums are correct, the problem either is not NAS vendor dependent, or different NAS brands are running the same O/S variant (e.g., busybox) and/or SMB version. It is also not clear that this is a macOS-specific problem either, as some *nix and Windows users have reported similar problems. However, most of these reports appear to be years old, so it is not clear the exact status of what is currently occurring.
I just wanted to throw this out there to see if anyone else has been recently experiencing SMB/NAS issues, or "chunk" or metadata (tags) issues on files written to NAS over SMB.
If you have your media library on NAS, and you use SMB to connect to your NAS, I'd like to request that you to run an experiment on a few known good CDs:
1) Set dBpoweramp to either: "when ripping: rip directly to final filenames" or "when ripping: rip to tmp files, rename album at end"
2) Rip a few CD with a mixture of short and long tracks. If you can rip from multiple drives simultaneously, that would be another good test.
3) Do the CDs consistently rip without error (that is, no "chunk" or other errors)?
4) Is the metadata consistently valid (that is, no garbled tags or missing tracks)? (You can use PerfectTUNES to check)
If you have problems, I'd like to know:
1) What your dBpoweramp "when ripping" setting are
2) What computer (Mac or PC) and O/S release you are using
3) What NAS and firmware release you are using
4) Whether the NAS was heavily I/O loaded (high activity) when the error(s) occurred
If you have had corruption problems (chunk, metadata, etc.) in the past, and you know the environment in which that occurred, I'd also like to know the circumstances under which it occurred.
For the record, when ripping CDs using "rename album at end", I was getting about 15% of my CDs having bad rips, and most errors seemed to occur when I was ripping 2 or 3 CDs simultaneously. I have had errors when tee-ing to a log occur twice in a probably a hundred different script runs, and both failures were when there was several other processing hitting the NAS at the same time--but, I have insufficient evidence one way or the other whether load is part of the issue.
Any insights anyone can provide will be GREATLY appreciated!
Finally, I am deliberately NOT saying what NAS or macOS version I am running, as I don't want to unintentionally prejudice the experiment. Please post failure results here.
Thanks.
Jon K
For weeks now, I have been trying to track down why I get "Chunk" errors ("Chunk is out of file") when ripping to NAS, but not when I rip to a local temp dir, then move the files to NAS. Well, I now have some preliminary answers--but, it is not clear whether this is a macOS issue, an SMB issue, or a NAS issue, and whether it is only affecting macOS or impacts Windows (and *nix) as well.
My first clear indication that corruption when writing to NAS is an actual problem occurred when I was running a script to check for organization issues in my music library. The script was executed by:
bin/validityCk.sh 2> logs/vc.err | tee logs/vc.log
and the script, my music library, and all dirs were on my NAS, as was my macOS current working directory in Terminal. As the script ran, I saw normal results displayed via tee. However, later in the day when I went back to review the vc.log file, all but the last few lines were nulls--a few thousand nulls, about one for every line which should have been in the log.
Searching, I find various reports of similar problems on numerous developer forums. If the forums are correct, the problem either is not NAS vendor dependent, or different NAS brands are running the same O/S variant (e.g., busybox) and/or SMB version. It is also not clear that this is a macOS-specific problem either, as some *nix and Windows users have reported similar problems. However, most of these reports appear to be years old, so it is not clear the exact status of what is currently occurring.
I just wanted to throw this out there to see if anyone else has been recently experiencing SMB/NAS issues, or "chunk" or metadata (tags) issues on files written to NAS over SMB.
If you have your media library on NAS, and you use SMB to connect to your NAS, I'd like to request that you to run an experiment on a few known good CDs:
1) Set dBpoweramp to either: "when ripping: rip directly to final filenames" or "when ripping: rip to tmp files, rename album at end"
2) Rip a few CD with a mixture of short and long tracks. If you can rip from multiple drives simultaneously, that would be another good test.
3) Do the CDs consistently rip without error (that is, no "chunk" or other errors)?
4) Is the metadata consistently valid (that is, no garbled tags or missing tracks)? (You can use PerfectTUNES to check)
If you have problems, I'd like to know:
1) What your dBpoweramp "when ripping" setting are
2) What computer (Mac or PC) and O/S release you are using
3) What NAS and firmware release you are using
4) Whether the NAS was heavily I/O loaded (high activity) when the error(s) occurred
If you have had corruption problems (chunk, metadata, etc.) in the past, and you know the environment in which that occurred, I'd also like to know the circumstances under which it occurred.
For the record, when ripping CDs using "rename album at end", I was getting about 15% of my CDs having bad rips, and most errors seemed to occur when I was ripping 2 or 3 CDs simultaneously. I have had errors when tee-ing to a log occur twice in a probably a hundred different script runs, and both failures were when there was several other processing hitting the NAS at the same time--but, I have insufficient evidence one way or the other whether load is part of the issue.
Any insights anyone can provide will be GREATLY appreciated!
Finally, I am deliberately NOT saying what NAS or macOS version I am running, as I don't want to unintentionally prejudice the experiment. Please post failure results here.
Thanks.
Jon K