Batch Ripper Requests / Suggestions

[EliC]

-more options in ripper to better support batch ripper handling of problem discs; skip tracks with > X or > X% individual bad frames in secure ripper settings, skip track if any pass is >X times realtime, more graceful rip canceling

-Rip Sessions, option to skip discs already in library (based on TOC, AR DiscID,... which ever has the least collisions - I don't know)

http://forum.dbpoweramp.com/showpost.php?p=69360&postcount=4

-skip commercial meta-data lookups on bad rips (to avoid paying for meta data on discs that cannot be ripped)

-more information from the ripper passed to the batch system: disc's metadata, accuraterip and secure "status" flags

http://forum.dbpoweramp.com/showpost.php?p=73140&postcount=37

http://forum.dbpoweramp.com/showpost.php?p=72039&postcount=172

[EliC]

Re: Batch Ripper Requests / Suggestions

-set max number of simultaneous rips.

[Porcus]

Re: Batch Ripper Requests / Suggestions

A few of my wishes in http://forum.dbpoweramp.com/showpost...3&postcount=21 are Sony mediachanger-specific, but generally, I would hope for more reject criteria like "is CD in AccurateRip?" or "not audio-only".

[EliC]

Re: Batch Ripper Requests / Suggestions

-Option to NOT attempt to rip HTOA discs (why pay for the look up), and an HTOA flag so the discs can be separated and manually ripped.

[Porcus]

Re: Batch Ripper Requests / Suggestions

Wouldn't it be trivial to implement -- in dBpoweramp itself, not only Batch Ripper -- a copy-all-files-to-folder for non-audio content? It could be in a sub-folder, so you wouldn't have to worry about someone putting a wav file as "non-audio data".

This and any HTOA flagging (preferably auto-ripping though ...) are top of my wishlist.

[EliC]

Re: Batch Ripper Requests / Suggestions

-option to limit the number of simultaneous rips. This should not delay robots from placing discs and readying drives, just delay the rip. This will limit CPU load which could be very high with many drives ripping at once on a single core CPU.

[bhoar]

Re: Batch Ripper Requests / Suggestions

-option to limit the number of simultaneous rips. This should not delay robots from placing discs and readying drives, just delay the rip. This will limit CPU load which could be very high with many drives ripping at once on a single core CPU.

In addition, this may provide some benefit to situations where multiple "difficult" discs are ripping in parallel even on a multicore system.

Even better would be support for having two limits: number of simultaneous coreconverter processes (audio compression) and number of simultaneous cdgrab processes (ripping).

I'd even support this being a registry-only feature set initially, for use in specialty situations, if you want to avoid the clutter or GUI work for it.

-brendan

[EliC]

Re: Batch Ripper Requests / Suggestions

shouldn't you take that one step further and have an option to limit the number of simultaneous ultrasecure rips? I imagine the burst rips are rather non-cpu intensive, but the ultrasecure can be quite taxing. So if you are ripping 4 discs and one is in ultra-secure mode, another would not enter ultrasecure mode until the first is finished but the other 2 drives could continue ripping in burst if no other limits are exceeded.

[bhoar]

Re: Batch Ripper Requests / Suggestions

shouldn't you take that one step further and have an option to limit the number of simultaneous ultrasecure rips? I imagine the burst rips are rather non-cpu intensive, but the ultrasecure can be quite taxing. So if you are ripping 4 discs and one is in ultra-secure mode, another would not enter ultrasecure mode until the first is finished but the other 2 drives could continue ripping in burst if no other limits are exceeded.

I suppose, but that would involve delving into the cdgrab.exe behavior, having multiple cdgrab.exe processes talk to eachother about what decisions they are making, etc. Limiting it to our previous suggestions would involve simpler code changes to the batch ripper, assuming my understanding of the architecture is correct.

-brendan

[EliC]

Re: Batch Ripper Requests / Suggestions

I can't believe I forgot this:

Option to skip non-pressed CDs and flag non-pressed CDs.

I think this is important for professional companies that do not want to rip pirate/burned discs. And for personal users that do not want to accidentally pollute there collection with burned discs.

[bhoar]

Re: Batch Ripper Requests / Suggestions

-option to limit the number of simultaneous rips. This should not delay robots from placing discs and readying drives, just delay the rip. This will limit CPU load which could be very high with many drives ripping at once on a single core CPU.

Spoon:

Based on some recent tests, I've come up with another factor (or rather, a more explicit argument of a factor touched on above) in our argument for a batch ripper feature to allow the user to set a cap on the number of simultaneous rips (or cdgrab.exe tasks).

Implicit in the feature request would be that the limit was not tied to the number of active drives and that the drives would be handled in a first-ready, first-serve order.

Anyway, on to the argument...it's really about reducing the disc cycle time while ensuring that the ripping machine is not overburdened with Interrupts, IO, RAM usage or CPU usage.

For example, one simple counter-argument to our position could be "If you want to limit it to say, one rip per CPU, or why not just ensure you only have one drive selected per CPU installed (or whatever you are basing your maximum metric on)?".

In response, I'd say that's non-optimal for the following reason: the unload/load cycle time for robots, including no- or low-IO time periods such as disc-recognition and meta-data lookup, can be 20-40 seconds, or even greater for some models. That's a loss of potentially precious ripping IO bandwidth. By allowing the pre-positioning of discs in more drives than the batch ripper is set to rip from simultaneously, you can reduce the apparent cycle time to close to zero without overburdening the system.

The limit that the operator imposes could simply be based on experience when an operator notices the system gets "bogged down", for whatever reason (IO/RAM/CPU), when >n rips were running to the point where the throughput effects became negative for additional parallel rips. E.g. the operator may even discover the best setting might even differ depending on the codecs chosen.

My suggestion would be to put the rip queue control right after the batch ripper metadata lookup and processing which, I think, is right before cdgrab.exe task is launched.

Assuming discs in good condition, this way you are able to pre-position resources to keep the machinery fed to the maximum and doing 100% of the job it is capable of.

Thoughts?

-brendan

[bhoar]

Re: Batch Ripper Requests / Suggestions

So, say I had four of the Microboards MicroOrbit devices with custom-added USB and serial interfaces (or the Kodak Kiosk variant with the 5-disc or less detection hardware and the interfaces factory built in) connected at once...

...but I was limited to using them on a single (or even dual) CPU system. I would probably want to configure the batch ripper for a maximum of two simultaneous ripping threads, especially if I were using a codec with high processor demands, right?

I wouldn't want to have two of the devices sit unused, which is currently my only choice.

Example #1...

The average cycle time using these robots is as follows...

Physical steps: 12 seconds.

[Physical represents: tray open, robot grab, tray close, robot accept/reject, tray open, robot insert, tray close.]

And physical + spin-up + CD recognition* by both ULCLI and batch ripper + metadata step (with no providers configured): 28 seconds.

If I were using four devices with a limit of two ripping threads, that idle time would all but disappear. If only using two devices, it would impact throughput noticeably.

Example #2...

Imagine, for a moment, that one or more of the metadata providers was experiencing web connectivity problems (or I was having more local packet loss causing delays). Depending on the timeout code in the batch ripper, this could also induce additional delays of 30 seconds or more after each inserted disc.

Again, if I were using four devices with a limit of two ripping threads, that wait time would all but disappear. Not so using two devices.

-brendan

* can be longer for other disc types...esp. if the drive does not recognize the discs.

[EliC]

Re: Batch Ripper Requests / Suggestions

-Option to NOT attempt to rip HTOA discs (why pay for the look up), and an HTOA flag so the discs can be separated and manually ripped.

I would like to bring this request up again. Since batch ripper is now final and there is currently no way to Auto-Rip HTOA, I would like to be able to separate these discs so the HTOA track is not missed by ripping with the batch ripper.

[EliC]

Re: Batch Ripper Requests / Suggestions

-skip commercial meta-data lookups on bad rips (to avoid paying for meta data on discs that cannot be ripped)

A possible work-around for this:
As previously suggested, allow ripper to send albums to Accurate or Inaccurate/Re-Rip directories. Since the files on the bad rips would already be in the re-rip directory with the meta-data you would not need to lookup the disc again, just re-rip the audio after the disc has been repaired.

[bhoar]

Re: Batch Ripper Requests / Suggestions

From the previous (locked) thread, Eli said:

"-Separate accurately ripped and inaccurate directories: When a disc is ripped accurately the files are placed in the normal directory. When one or more tracks on the disc are inaccurate the user would have the option to move all of the disc, or just the inaccurate files to an incomplete or inaccurate directory separate and distinct from the main library. If in the future the disc is repaired or a different copy of the same pressing is ripped and all of the files are ripped accurately dB moves the album over to the accurate directory."

I'd like to see something similar, but perhaps framed in such a way that it is easier for spoon to implement? This would be a general ripper feature, not just limited to the batch ripper.

Really, my ideal process would work something like this...

1. An option can be turned on to sort each CD's output by the quality result of ripping (accurate, secure, insecure/unfinished) vs. a single output path.

e.g. At the end of the cdgrab.exe (ripper) task, it, or a dsp effect, goes through the in-memory record of this CD and decides whether to move the CD's directory from Working to either Secure or Accurate (or not at all).

There are three sister directories defined, perhaps all required to be on one volume (for performance reasons - a move within a volume does not involve reading/writing the entire file) or in one particular sub-tree:

- Working (or Incomplete) directory - all ripping happens here initially and stays here if the rip is either aborted or *any* track is insecure.

- Secure directory - if all tracks were ripped secure but one or more track was not in Accuraterip or did not match Accuraterip, the files get moved here.

- Accurate directory - if all tracks were ripped accurately, the files get moved here. That is, this is always the final resting place for known good rips. Perhaps the threshold is adjustable (e.g. requires Accurate (3) or higher).

Spoon may want to implement the setting via a) custom code and a new GUI feature or b) some sort of DSP effect (perhaps the DSP action can check to see if it is running on the final track before acting) or c) a pathname variable (hard to see how that would work). The important part to note is that the final path is only known when the CD has finished (or aborted) ripping, so that needs to be kept in mind in terms of where the appropriate option comes in.

2. An additional add-on would be an option to have the ripper automatically skip completed tracks found in the Working directory. Possibly even only skip accurate or secure tracks found in the working directory.

-brendan