Batch Ripper Universal CLI Usage Guide

[Spoon]

(available shortly)

[bhoar]

Batch Ripper Universal CLI Usage Guide - Part I

The Universal Loader CLI (ULCLI) Tool Usage Guide for ULCLI v0.88
aka "How to create your own driver set for dbpoweramp's Batch Ripper for Fun and Profit."

------------
---Part I--- ULCLI Architecture, Parameters, Device Exploration...and a brief example.
------------


--A-- Warranty and Licensing

Warranty: There is no warranty expressed or implied. In addition, future versions may differ in license, functionality and features.

License: The ULCLI is currently licensed to spoon for use and distribution with dbpoweramp's batch ripper.

In addition to the above license: having downloaded the program with dbpoweramp, you may also use the ULCLI outside of dbpoweramp to conduct your own personal or private business affairs in any way you like. For example: if you were to write a windows script that integrates a loader with backup software to pull data from your local office hard drives to BD-R discs, it is permitted to incorporate the ULCLI in your script.

However, unless you are spoon, you do not have the right to redistribute and/or sell the ULCLI software without obtaining the explicit permission of the author (me). My email address is available in the ULCLI usage statement should you need to contact me directly. As expected, there are some grey areas between the open use license and the distribution restriction (e.g. "can I distribute it to all staff in my company across multiple offices?"), and I'm always willing to discuss.



--B-- Keep in mind...

- This guide is based on version v0.88 of the ULCLI. As such it is considered an alpha and later versions may require changes to your device-specific scripts, though an attempt will be made to minimize the changes you will need to make.

- Always check the version of the tool (by running the executable in a command window with no parameters) to ensure that you are using the right version of the guide with the right version of the tool. It will show up at the top of the usage statement (you may need to scroll back a bit to see it.).

- For the purposes of this tool, "loader" can refer to any serial-port controlled "autoloader" or "duplicator" device that handles optical discs with robotics. It generally does not refer to "changers", however.

- Serial controlled loaders do differ in several important ways that will have a bearing on the approach you take. Even if the hardware appears to follow the industry standard command set (commands: C, V, I, A, G, R, responses: X, E, ?), the following areas may require the drivers to diverge: location of the input, output or reject areas with respect to drive trays; placement of drives with respect to other drives; disc stack auto-management features; startup procedures; collision or dropped disc recovery procedures; picker reliability; firmware or macro version behavioral differences; etc.

- The ULCLI handles each drive using a different invocation of the program. This means there isn't a central loader "handler", but instead, often several invocations of the ULCLI have to arbitrate among themselves "who gets to go do what and when". This arbitration is handled via several kinds of "locking" files stored in the normal user-specific temporary directory. There are several locking approaches built into the ULCLI: e.g. the com port serves as a primary device lock, while "bind"ing to a drive helps with drive column tracking for implicit tray collision avoidance. These locks are there to ensure that unload/load/reject sub-tasks are not interleaved in such a way as to physically interfere with each other.

- Performance: The locking mentioned above can get complex. Sometimes the simple approach does not allow the ULCLI to maximize throughput as much as it can. Remember each invocation of the program is separate and has no knowledge of what additional request may come next from the application that starts another invocation. As of 0.88 performance is not fully optimized, having taking the approach of erring on the side of device safety (mostly due to the lack of error handling as noted below). So, there may be some loading delays that appear inexplicable to you. They are there for a reason. I will be working on addressing throughput as best that I can before wrapping up 1.0, with the caveat mentioned above of having to safely handle whatever parameters come in from the calling applications.

- As of 0.88, there is no hardware error handling when an error response or time-out is detected: the ULCLI exits immediately if the device either returns an error code or does not respond within the time-out parameters. Within the context of dbpa, the batch ripper will normally re-invoke the ULCLI in this situation. The lack of hardware error handling in the ULCLI will be remedied before the 1.0 release is finalized, including the addition of a "wait for operator" notification feature when unexpected errors or time-outs are encountered. Because of the above mentioned lack of comprehensive device response error handling: if there is a dropped disc, picker->tray collision, etc., then the batch ripper and ULCLI will probably continue to retry loading the drives. In some circumstances, this may lead to disc or even hardware damage. With that said, in my testing it has worked fine with various pieces of equipment that are in good repair when called from dbpa's batch ripper.



--C-- ULCLI's Parameters.

From the usage statement generated if you run the executable with no parameters.

C:\Program Files\Illustrate\dBpoweramp\BatchRipper\Loaders\Amtren or Discmakers Elite Micro\Load>load.exe

Universal Loader CLI  v0.88
  Configurable for any serial-controlled CD/DVD/disc duplicator/autoloader.
  Copyright 2007 Brendan G Hoar (xxxxxxx.xxxx@xxxxx.com). All Rights Reserved.
  Serial port interface by Martin Gibson (martin, AutoIt script forums).

  Licensed for redistribution with Illustrate's dbpoweramp Batch Ripper only.

Hacks/Debugging (can be anywhere on command line):
  --debug (put ULCLI in a 'DOS window' so you can monitor the process)
  --discstatusdebug (outputs drive/disc status continuously, ignores rest of parameters)

Parameter-Settings (can be anywhere on command line):
  --ComPort=COM## (defaults to COM1)
  --ComSpeed=##### (defaults to 9600)
  --ComSettings=XXX (defaults to 8N1)
  --ComHandshake=XXXX (defaults to None)
  --ComPortAccessTimeout=### (max wait time, in s, to open port-default=480)
  --ComPortAccessTimer=## (how often, in s, to retry port in use-default=1
  --drivebank=# (sets current numeric bank identifier:  1 or 2)
  --spacestring= (default is _, change if you need to use this char)
  --enterstring= (default is \M, change if you need to send this substring)
  --colonstring= (default is `, change if you need to send this substring)
  --pollstring= (set this to query string if robot requires it-default="")
  --polltimer=# (how often, in s, to poll/check serial port-default=1)
  --commandtimeout=## (default, in s, if %timeout% unused-default=30)
  --commandminimumtimeout=# (default, in s, if %minimumtimeout% unused-default=3)
  --rejectsmax=## (pause for operator every ## rejects-default=10)
  --olddrives (use if drives cannot report tray position via Medium Status CDB)
  --logfile= (redirect output to a log file as specified use quotes around path)

Parameter-Simple Commands (invoke order matters, as these are all queued & processed):
  --drive=XX (sets drive letter:  D: or E:, etc.)
  --open (open drive tray - requires ULCLI to grab com port, if not already open, before opening tray)
  --close (close drive tray - does not require open com port)
  --closeall (close all drive trays)
  --bindtodrive (tells other ULCLI instances that drive bank is busy)
  --unbindtodrive (tells other ULCLI instances that drive bank is free
  --bindresetall (frees all drive banks)
  --comportconnect (explicitly open com port to lock robot to this instance)
  --comportrelease (allow other instances to use robot to service other drive banks)
  --rejectsiterate (increments counts of rejects in reject stack
  --rejectsreset (clears count of rejects in reject stack)
  --beep{=#} (# is hertz)
  --exit{=#} (# is return code)
  --sleep{=###} (in s-default=1)
  --sleeprandom{=###} (in s, sleep between 1 to ###-default=1)

Parameter-Complex Commands (invoke order matters, as these are all queued & processed):
  --command=%sendstring%{:%expectedresponse%{:%timeout%{:%minimumtimeout%{:%failureresponse%}}}}
  --commandbank1=... (as above, but only executes when drivebank=1)
  --commandbank2=... (as above, but only executes when drivebank=2)
    note: use underscore for space and \M for carriage return in command strings
          unless you changed them via --enterstring= or --spacestring=.
  --ondiscnotreadyskip=# (if no or non-readable CD in drive, skip forward # commands)
    note:this parameter will be deprecated in future versions which will have parameters
         which allow for robot error handling and flow-of-control changes.

This program uses the user's temporary directory to coordinate robot/drive access.  Do not
clear your temporary directory while the ULCLI is running, or you may risk hardware
damage.

C:\Program Files\Illustrate\dBpoweramp\BatchRipper\Loaders\Amtren or Discmakers Elite Micro\Load>

Additional notes on points that may not be covered well above:

1. The load.exe, unload.exe, reject.exe, pre-batch.exe and post-batch.exe programs in the device driver folder are actually all the same executable (normally called ulcli.exe). They ignore what they are named, and only pay attention to the rest of the command line (the parameters).

2. --pollstring is only necessary if your loader requires constant poll commands to return status (e.g. Primera Composer).

3. "Drive Binding": each drive is handled by an additional copy of the program and needs to communicate to the other copies that an "unload/reject then load" cycle is being performed in a particular drive column and therefore no other drives should be serviced until it is complete. Again, this helps to prevent multiple open trays from conflicting. Rather than lock the entire loader, we're only locking a column of drives, so that other columns can be serviced if there is a disc spin-up delay (e.g. testing that a disc was loaded and recognizable before OKing the return to the calling application can take some time). Binding is not necessary if you have a single drive or if all your drives are located such that you don't care what other trays might be opened when loading or unloading discs.

4. Please read through all of the above carefully and take notes. Did you? If so, you would have picked up that while --close doesn't care about the serial port, --open always makes sure the current process has exclusive access to the serial port, and will try and retry to get it, so as to prevent multiple invocations from opening trays on top of one another.

5. If you have a multi-drive loader where multiple open drive trays can interfere with disc loading/unloading/rejecting, then you must use the "bindtodrive" and "unbindtodrive" parameters appropriately. This will be covered in part II.

--D-- Some issues and questions to explore while testing your device:

1. On PCI (PC-Inside or PC-Integrated) loaders, I generally install a DPST (or DSDT wired as a DPST) switch between the robot controller board and the power supply, which allows me to perform a full power-off/power-on hardware reset of the loader without having to interrupt the PC session that also depends on the power supply.
2. Every loader I've used works at 8N1 with no flow control expected. The speeds differ, but are typically 9600 (or 38400 for more recent units), with 14400 and 19200 relegated to Primera and RImage devices.
3. On some machines, the serial port may not be the usual DB-9 female connector. It could be a DB-9 male connector, it could be null-modem wired (or unexpectedly not), or it could be an internal two or three pin header. You may need to modify the wiring or disconnect the robot board from a duplicator controller board. Always keep notes, so that you can return the hardware to the original configuration, if necessary. I've found having a lot of low-profile db-9 gender-changer and null-modem adapters helpful.
4. Most command sets are single character and in "immediate" mode where as soon as the command is received, it is acted on. Primera and RImage are different in this regard, see the hyperdiscs wiki for more info.
5. Does the device need a reset or other command before it works after power-on? Put that in the pre-batch.
6. After a disc pick/place error or picker movement error, does the device need a reset or other command before it will properly respond to the next normal request? since we have no error handling, it can prove useful to always force a reset (e.g. send a "C"alibrate) command at the beginning of each invocation.
7. Does the device need a reset or other command before certain commands at all times? There are some devices out there that will only "I"nsert if the last command was a "C"alibrate.



--E-- Some general hardware rules:

1. Never move the mechanism by hand while power is applied. Often times stepper motors are full energized when stopped and resist externally generated movement, which means that moving parts by hand puts a larger amount of stress on components and can lead to snapping off or cracking parts.
2. Some mechanisms can be moved (gently!) while the power is not applied. When doing so, you should avoid applying unnecessary torque. Example: when moving a picker up and down manually, do not push it at the business end, but rather grasp it as close to the vertical column as possible so no undue stress is applied to either the picker mount or the column mount.
3. Powered-off manual manipulation can be useful for putting the device in the safest position before transport, or recovering from a hardware fault that the power-on reset or the reset command fail to address (such situations can arise with older firmware/macros).



--F-- An example of how to identify the device commands useful for the ULCLI...

Stepping away from the ULCLI for a bit. Instead it's time to fire up Hyperterminal or, better yet, putty if you know that tool.

While this example avoids some of the complex issues that come up when devices have multiple drives in a vertical column (and even potentially multiple columns), it shows the core process of using the ULCLI to create a driver directory for dbpa batch ripper.

For this example, I will use the most basic, yet still on the market, serial-controlled robotic loader I have access to.

The device looks like the following picture and may be found under several names:



Amtren Flexwriter I
Discmakers Elite Micro
Discmakers Micro Ultra
MF-Digital Ripstation 7601X
Ripfactory Ripstation MD1XC

Note: The rear panel I/O ports have changed over time. From what I've seen the older device have two ports we care about, a 9-pin RS-232 and a Firewire->IDE. The newer ones replace those with a single rear USB connector that leads to an internal USB Hub which connects to internal USB->Serial and USB->IDE devices.

We are ignoring any printer-control or USB passthrough ports, of course.

The core command set, found via trial and error (it's helpful to use the reset command, once found, between tests), is very simple:

C - (re)Calibrate (aka Reset)
V - Send me the "Signup String" (identifies the device to the software)
I - Insert (move from input to drive tray)
A - Accept (move from drive tray to output)
G - Get (pick from drive tray and hold)
R - Reject (with disc already held in picker, place on table underneath position of drive tray, requiring disc tray to be closed)

Responses are:
X - Success
E - Failure/Error
? - command not recognized

Some of the larger loader devices have complex macros (loadable command sets) and probing these units can leave the units in modes where the familiar commands stop functioning. Be prepared to power the loader off, then back on, to recover from such situations while investigating the command sets.

If you had derived the command list above from experimentation, you would have noticed how only two of the core disc movement related commands, Insert and Accept (but not Reject), are single step commands. This is because there is no need to interact with the disc tray except before or after those two commands. However, a disc rejection requires two commands with an interceding disc tray action: a "G" (Get) to pick the disc, a closing of the tray, and a "R" (Reject) to place the disc below the tray.

As I mentioned in the first section, finding these instances where there can be physical interference between drive trays and the picker (or access to other drive trays or bins) is key to putting together a good driver.



--G-- The simplest invocation example...let's load a disc using a handcrafted ulcli invocation of the load.exe found in this driver directory. Let's pretend, again, that we have one of the most common pieces of loader hardware, a Discmakers Elite Micro:

load.exe --comport=com1 --comspeed=9600 --drive=D: --open --command=I:X:30:5:E --close

What does the above mean?
- Run the executable
- Set the com port
- Set the com speed
- Set the drive we want to work with
- Open the drive tray (making sure that we get a lock on the serial port first)
- Send "I" to the com port, keep an eye out for success ("X") or failure ("E"), and don't act on results for at least 5 seconds. IF "E" is seen or there was at least 30 seconds with no response exit the load.exe entirely (after waiting at least 5 seconds to act). IF "X" is seen and at least 5 seconds have elapsed, move to the next parameter.
- Close the drive tray (ignoring the state of the serial port connection)
- Exit the load.exe entirely.



-----

And so ends Part I.

Part II, which covers both a breakdown of the Elite Micro driver and one of the larger multi-drive units should be available in several days.

[bhoar]

Re: Batch Ripper Universal CLI Usage Guide - Part I

Just an update - nearing a workable 1.0 release, once that is done, I will continue with the guide.

As of today, I've added support for:

- Flow of control enhancement using labels and skip commands
- Setting and using variables across ULCLI processes (including scope specification)
- Conditional statements that harness the two items above
- Notifications and handling of responses
- Passing messages back via a text file to the calling program
- Setting return codes

Some of the items above were necessary to support proper start and end of batch behaviors, handle errors reported by the hardware, allow the operator to make decisions on whether to continue or abort, tell the batch ripper when the operator elects to stop ripping and handle hardware that needs the ULCLI to keep track of stack status and drive positions.

Since I cannot modify the post above, here is the current usage statement.

Universal Loader CLI  v0.96
  Configurable for any serial-controlled CD/DVD/disc duplicator/autoloader.
  Copyright 2007-2008 Brendan G Hoar (xxxxxxx.xxxx@xxxxx.xxx). All Rights Reserved.
  Serial port interface by Martin Gibson (martin, AutoIt script forums).

  Licensed for redistribution with Illustrate's dbpoweramp Batch Ripper only.

"Commands" prefixed with a * below are pre-scanned and treated as global settings.
All others are executed in order as discovered, i.e. some settings can be adjusted
during processing. Commands with multiple parameters use a colon as a delimiter
optional parameters are shown using square brackets.

Global Settings
* --spacestring= (default is _, change if you need to use this char)
* --enterstring= (default is \M, change if you need to send this substring)
* --colonstring= (default is `, change if you need to send this substring)
* --separatorstring= (default is :, change if you need to send this substring)
* --logfile= (redirect output to a log file as specified use quotes around path)

Serial Ports
* --ComPort= (defaults to COM1)
* --ComSpeed= (defaults to 9600)
* --ComSettings= (3 chars: bits, parity, stop bits - defaults to 8N1)
* --ComHandshake= (None, Hard, Soft - defaults to None)
* --ComPortAccessTimer= (how often, in s, to retry port in use-default=1
  --ComPortAccessTimeout= (max wait time, in s, to open port-default=480)
  --comportconnect (explicitly open com port to lock robot to this instance)
  --comportrelease (allow other instances to use robot to service other drive banks)

Drives
  --drive= (sets drive letter - D: or E:, etc.)
* --drivebank= (sets current numeric bank identifier:  1 or 2)
    'drivebank' groups drives where open trays can interfere with access
* --olddrives (use if drives cannot report tray position via Medium Status CDB)
  --open (open drive - open requires ULCLI to grab com port for safety, if not already open)
  --openblind (as above, but don't wait for spin down - use only at end of batch) 
  --close (close drive and wait for spin up - does not require open com port)
  --closeblind (as above, but don't wait for spin up - always use --sleep=2 after this) 
  --closeall (close all drive trays)
  --bindtodrive (tells other ULCLI instances that drive bank is busy)
  --unbindtodrive (tells other ULCLI instances that drive bank is free)
  --bindresetall (frees all drive banks)

Robots
* --commandtimeout= (default, in s, if %timeout% unused-default=30)
* --commandminimumtimeout= (default, in s, if %minimumtimeout% unused-default=3)
* --pollstring= (set this to query string if robot requires it-default="")
* --polltimer= (how often, in s, to poll/check serial port-default=1)
  --command=sendstring[:expectedresponse[:timeout[:minimumtimeout[:failureresponse]]]]
  --commandbank1=... (as above, but only executes when drivebank=1)
  --commandbank2=... (as above, but only executes when drivebank=2)
    use underscore for space and \M for carriage return in command strings
    unless you changed them via --enterstring= or --spacestring=.
    You may skip parameters, but you must use the right # of colons e.g. --command=I:X:::E

Special Reject Count Handling
  --rejectsmax= (pause for operator every n rejects-default=10)
  --rejectsreset (clears count of rejects in reject stack)
  --rejectsiterate (increments counts of rejects in reject stack

Basic Flow of Control
  --exit[=#] (# is return code)
  --sleep[=#] (in s-default=1)
  --sleeprandom[=#] (in s, sleep between 1 to #-default=1)
  --label=labelname (used as a target for skip commands, must begin with a letter)
  --skip=sv (unconditionally skips to a label or relative position)
    sv, or 'skipvalue' - can be a label name, or a +/- numeric value
  --skipback (returns to the last place skipped from - only works for last skip, no nesting)
  --ifcommandfailskip=sv (if last command was not successful, skip based on sv)
  --ifcommandtimeoutskip=sv (if last command timed out, skip based on sv)
  --ifcommandfailortimeoutskip=sv (if last command failed or timed out, skip based on sv)
  --ifdiscnotreadyskip=sv (if no or non-readable CD in drive, skip based on sv)
  --ifdiscnotcdromskip=sv (if writable CD or non-CD in drive, skip based on sv)
  --ifdiscnotcdskip=sv (if non-CD in drive, skip based on sv)
* --discspinuptimeout=# (defaults to 60 seconds, used for the --ifdisc items above)

Variables
  --clear=name[:scope] (clear a variable)
    scope of variable is optional - can be global, bank or drive.  drive is default
    it determines which other ULCLI processes handling other drives can see the variable.
  --clearall[=scope] (clear all variables, default scope is drive)
  --set=varname:value[:scope] (set a variable)
* --varname=value (special shortcut for --set=varname:value, always in a scope of 'drive')
    Any variable set via the above two methods can be used by enclosing in curly braces.
    e.g. --set=inputstack:2:global
    e.g. --command=_MOVE{inputstack}:$S.{inputstack}:15:5
    e.g. --notify="An error occurred loading from stack {inputstack}."
    You can also also return a subset of the variable: {VARNAME,START,LENGTH}.
    e.g. --notify="An error occurred loading from drive letter {drive,1,1}."

Implicitly set Variables (that is, set by ULCLI)
  {drive} = drive letter
  {drivebank} = drive bank (1 or 2)
  {comport} = e.g. COM2
  {notifyreturncode} = button pressed by operator during last --notify= call

Complex Flow of Control using Variables
  --ifsetskip=name:value:sv[:scope] (use variables as execution conditionals)
  --ifnotsetskip=name:value:sv[:scope] (use variables as execution conditionals)

Notifications
  --notify=message[:title[:type[:timeout]]] (notify operator with message and beep)
    Message and Title are double quote-enclosed strings (no colons allowed), timeout is in s)
    Button press sets the variable {notifyreturncode})
    Types can be:
      0 - OK
      1 - OK and Cancel (the default)
      2 - Abort, Retry, and Ignore
      3 - Yes, No, and Cancel
      4 - Yes and No
      5 - Retry and Cancel
      6 -  Cancel, Try Again, Continue 
    The {notifyreturncode} set by Operator's button press will be:
      1 - OK
      2 - CANCEL
      3 - ABORT
      4 - RETRY
      5 - IGNORE
      6 - YES 
      7 - NO
      10 - TRY AGAIN
      11 - CONTINUE
  --beep[=#] (# is hertz)
* --messagefile=filepath in quotes (used to send messages back to calling programs)
  --message=text in quotes (written to the file specified in messagefile)


This program uses the user's temporary directory to coordinate robot/drive access.  Do not
clear your temporary directory while the ULCLI is running, or you may risk problems,
perhaps even hardware or disc damage.

[hagak]

Re: Batch Ripper Universal CLI Usage Guide

Since you mentioned in the other thread about the Nakamichi 5disc changers, would it be possible to use this to make a few of the Nakamichi drives work with the batch ripper?

[bhoar]

Re: Batch Ripper Universal CLI Usage Guide

Since you mentioned in the other thread about the Nakamichi 5disc changers, would it be possible to use this to make a few of the Nakamichi drives work with the batch ripper?

This is outside the scope of my ULCLI project as the movement mechanism isn't controlled over a serial-port (instead it is done via the scsi or ata bus). Perhaps spoon can look into "internal changer" support for drives such as these?

-brendan

[Spoon]

Re: Batch Ripper Universal CLI Usage Guide

We are not planning on adding support, there is nothing stopping anyone else, the loader methods are fully open.