Recursive Directory Scripting/Conversion

[rossw]

After looking around for 1/2 day I found the scripting capabilities of dMC.
Only 3 edits needed to this script to get it to work for you!
Assumes you have the right encoders in the right place.

Hope everyone finds this useful! Enjoy and ask questions.

-Ross Warren

dMC-RDC.vbs

'dMC Recursive Directory Conversion
'Script by: Ross Warren, [email]rossw@woodhome.com[/email]

'
' Only 3 Changes that are needed are below
'

Rootsource= "C:\Move\Temp\FLAC"
SourceDrive = "D:\"
DestFormat = "FLAC"

'
' No Edits Needed Below this line
'
Set fso = CreateObject("Scripting.FileSystemObject")
Set dMC = CreateObject("dMCScripting.Converter")

'dMC Specific Settings
dMC.VolumeNormalize = False
dMC.PreserveTags = True
dMC.DeleteSourceFiles = False
dMC.ConvertToFolder = True

ConvertSubfolders fso.GetFolder(SourceDrive)

Sub ConvertSubFolders(Folder)
		
    For Each Subfolder in Folder.SubFolders
	    ConvertSubFolders(fso.GetFolder(Subfolder))
 	Next
 		For Each objFile in Folder.Files
 		
		dMC.AddFromToFiles (Folder & "\" & objFile.Name), _
		(RootSource & "\" & objFile.ParentFolder.ParentFolder.Name & "\" _
		 & objFile.ParentFolder.Name & "\"  & fso.GetBaseName(objfile))
		 
	Next
	     Call dMC.GoConversion(DestFormat, True, False, True, False)
End Sub

[LtData]

Re: Recursive Directory Scripting/Conversion

Just a tip: when you post code like this, it is normally a good idea to enclose the code in "[code]" brackets to make it easier to read and copy/paste the code.

[rossw]

Re: Recursive Directory Scripting/Conversion

My first time posting code
Thanks for the tip. Readability is much easier with the formatting

-Ross

[donny]

Re: Recursive Directory Scripting/Conversion

very nice of you to share your work

[SpritHansi]

Re: Recursive Directory Scripting/Conversion

Nice script. But it is possible to get a progress window??

[DazBYorks]

Re: Recursive Directory Scripting/Conversion

Hi,

See this thread

Daz

[insysnet]

Re: Recursive Directory Scripting/Conversion

Thanks for the script - I've updated it and thought someone else might benefit.

I've changed the actually conversion to use the command-line rather than Active-X component - this allows the script to run on Windows 64-bit without forcing it to run in the 32-bit scripting system.

I've also added the ability to filter the source files by file extension (so it doesn't try and convert non-audio files).

Additionally the script can be run multiple times as new files are ripped/added to the source as it will skip files that have already been converted.

I am running this using Windows scheduler to automatically keep my library up-to-date each night.

Hope the code below helps someone else....

'dMC Recursive Directory Conversion
' 64-bit Windows Compatible
' Only Convert files that don't already exist
' Limit source files to specific extensions
' Create log file

'Updated Script By: Richard Ashford, richard@richardashford.com
'Original Script by: Ross Warren, rossw@woodhome.com

'
' Changes needed below

' Location of dbppoweramp converter exe
' IMPORTANT - remove the " (x86)" below if you are running on 32-bit Windows

ConvertorEXE   = "C:\Program Files (x86)\Illustrate\dBpoweramp\coreconverter.exe"

' Source location for files

Source         = "C:\Users\richard.ashford\Music\iTunes\iTunes Media\Music"

'  Leave FileExtensions blank for all files, otherwise separate extensions with |

FileExtensions = "M4A|MP3"

' Target Location for converted files

Target         = "C:\Users\richard.ashford\Music\Windows"

' Target Format for converted files

DestFormat     = "Windows Media Audio 10"
DestCodec      = "Windows Media Audio 9.2 Lossless"
DestSettings   = "VBR Quality 100, 44 kHz, 2 channel 16 bit VBR"
DestOptions    = "-vbr"

' File extension for new files

DestFileExt    = "wma"

' Location of Logfile to be created

LogFile        = "C:\Users\richard.ashford\Documents\convert.log"


'
' No Edits Needed Below this line
'
Set fso = CreateObject("Scripting.FileSystemObject")

Set wshShell = WScript.CreateObject("WScript.Shell")

Set actionFile = fso.CreateTextFile(LogFile, True)

actionFile.WriteLine "START | " & NOW()

ConvertSubfolders fso.GetFolder(Source)

actionFile.WriteLine "END   | " & NOW()
actionFile.Close

Sub ConvertSubFolders(Folder)
		
    For Each Subfolder in Folder.SubFolders
	    ConvertSubFolders(fso.GetFolder(Subfolder))
 	Next
 		For Each objFile in Folder.Files

		    FileExtension =  "|" & UCase(fso.GetExtensionName(objFile)) & "|"

		    If Instr("|" & UCase(FileExtensions) & "|", FileExtension) > 0 Or FileExtensions = "" Then

		      TargetFile = Target & "\" & objFile.ParentFolder.ParentFolder.Name & "\" _
			    & objFile.ParentFolder.Name & "\"  & fso.GetBaseName(objfile) & "." & DestFileExt

		      If fso.FileExists(TargetFile) Then

			' Ignore File - already created

		      Else

			wshcmd = ""
			wshcmd = wshcmd & """" & ConvertorEXE & """"
			wshcmd = wshcmd & " -infile="""
			wshcmd = wshcmd & Folder & "\" & objFile.Name
			wshcmd = wshcmd & """"
			wshcmd = wshcmd & " -outfile="""
			wshcmd = wshcmd & TargetFile
			wshcmd = wshcmd & """"
			wshcmd = wshcmd & " -convert_to=""" & DestFormat & """"
			wshcmd = wshcmd & " -codec=""" & DestCodec & """"
			wshcmd = wshcmd & " -settings=""" & DestSettings & """"
			wshcmd = wshcmd & " " & DestOptions
			wshcmd = Trim(wshcmd)

			wshShell.run wshcmd, 1, true

			actionFile.WriteLine wshcmd

		      End If

		   End If 		

	Next

End Sub

[iangrant]

Re: Recursive Directory Scripting/Conversion

This is great! - looks to be just what I was after.

I notice on the developers' scripting web page it says: 'In-house Scripting License: A dBpoweramp Reference license is required for each PC & concurrent scripting object. For example 1 PC handles scripting, it is a dual cpu system and 2 scripting objects are used side by side, 2 Reference licenses are required.'

Does the above paragraph mean that only 1 CPU will be used with a 'normal' dBpoweramp Reference installation?

I have a Reference licence, and also a RipNAS Essentials licence. I want to do my scripting on my server which has RipNAS on it. I could also put my dBpoweramp Reference on there as well. Naturally I want it to use both cores for conversion.

[Spoon]

Re: Recursive Directory Scripting/Conversion

It is a commercial restriction, ie a huge radio station who put dbpoweramp on a 16 core server and run 16x concurrent conversions, they would require 16 dbpoweramp reference licenses.

[iangrant]

Re: Recursive Directory Scripting/Conversion

So I've been using this script to convert my FLACs to MP4 and noted a couple of things:

1. The '-codec=' and '-settings=' in the above script seem superfluous - are they from an old version of CoreConverter.exe? The CLI docs suggest that all is needed is '-convert_to=' and then the codec or dMC options come straight after that. Mind you, I am not using WMA, maybe they are specific to that codec.

2. The above script doesn't make use of multi-CPU encoding as far as I can tell. Am I right in thinking that the multi-CPU aspect of dBpoweramp means spawning two (or more) instances of the converter, i.e. converting two files in parallel, rather than using two CPUs on one file?
In which case, does anyone with some better scripting skills than I know how to adjust this script so that it can run two instances of CoreConverter at the same time?

[Spoon]

Re: Recursive Directory Scripting/Conversion

-codec and -settings are not required.

2. You would spawn two or more concurrent encodings.

[iangrant]

Re: Recursive Directory Scripting/Conversion

I took it upon myself to write a new script that will take advantage of multiple CPUs!
It launches as many concurrent instances of CoreConverter.exe as you specify. It also replicates the complete directory structure from source to destination (I think the previous scripts only replicated it as far as two levels up).

Perl's more my thing, so requisites are: Perl (duh...), with the File::Find::Rule and Parallel::ForkManager modules.
The MP4::Info is optional, if you're not converting to M4A you might want to remove the code that uses this.

These are all easily available - I am using ActiveState Perl and the modules are available in the package manager.
To run as a scheduled task, use wperl.exe instead of perl.exe to avoid command boxes popping up (e.g. "wperl.exe ConvertMusic.pl"). You'll also need to run the task as "NT AUTHORITY\SYSTEM" user - other users will probably cause the CoreConverter.exe windows to pop up. If you're running on Windows Home Server as I am, then these windows appear on the WHS Console if there's no Remote Desktop window open.

use File::Find::Rule;
use Parallel::ForkManager;
use MP4::Info;

*blooper*
*blooper* ConvertMusic.pl
*blooper* Author: Ian Grant <ian -at- iangrant -dot- me>
*blooper*


*blooper*
*blooper*   Configuration options
*blooper*   =====================
*blooper* 

*blooper* Use forward slashes in place of backslashes for these paths
my $source_dir  = 'D:/shares/Music/Ripped CDs';
my $dest_dir    = 'D:/shares/Converted Music/Ripped CDs';

*blooper* Case of source extension is important. Don't include the leading period.
my $source_ext  = 'flac';
my $dest_ext    = 'm4a';

*blooper* Output codec and options.
*blooper* It's easiest to set the desired options in CD Ripper and then look up the CLI syntax in the registry:
*blooper* See the HKCU\Software\Illustrate\dBpoweramp\CDRipper\Profiles\(default) key (or whatever profile you configured)
*blooper* The 'CodecCLI_<codec>' and 'DSPEffects' strings contain the values needed. Leave any backslashes as they are.
my $dest_codec    = 'm4a Nero (AAC)';
my $codec_options = ' -cli_encoder="C:\Program Files\Illustrate\dBpoweramp\encoder\m4a Nero (AAC)\neroAacEnc.exe" -cli_cmd="-q .45 -ignorelength -if - -of {qt}[outfile]{qt}" -selection="0,4,0" ';
my $dsp_effects   = ' -dspeffect1="Volume Normalize= -mode={qt}rg{qt} -maxamp={qt}8{qt} -desiredb={qt}-0{qt} -adapt_wnd={qt}6000{qt} -fixed={qt}0{qt}" -dspeffect2="Bit Depth=-depth={qt}16{qt}" -dspeffect3="Delete Destination File on Error=" ';

*blooper* dBpoweramp options.
*blooper* The '-silent' flag is recommended to avoid overlapping progress output. Leave paths with backslashes.
my $dmc_options   = ' -silent -error="D:\shares\Converted Music\Error Log.txt" ';

*blooper* Log file. Use forward slashes in the path, leave blank to disable.
my $logfile = 'D:/shares/Converted Music/Conversion Log.txt';

*blooper* Set this to the number of CPUs/cores in your system, or 0 to disable multi-processing.
my $number_of_cpus    = 2;

*blooper* Location of dBpoweramp CoreConverter.exe (leave with backslashes here)
my $coreconverter_exe = 'C:\Program Files\Illustrate\dBpoweramp\CoreConverter.exe';



*blooper*
*blooper* There should be no need to edit anything below this line
*blooper* ========================================================
*blooper*
*blooper*

*blooper*
*blooper* Logging sub-routine
*blooper*
sub logmsg {
  if ( $logfile ne "" ) {   *blooper* log to file if the logfile is defined
    open LOGFILE , ">>$logfile";
    print LOGFILE "$_[0]\n";
  }
    print "$_[0]\n";   *blooper* also print to screen (unless running under wperl.exe)              
}


*blooper*
*blooper* Find all source files
*blooper*
my @srcfiles = File::Find::Rule->file()
                            ->name( '*.' . $source_ext )
                            ->in( $source_dir );


*blooper*
*blooper* Output header info
*blooper*
logmsg "==================================";
logmsg "dMC recursive directory conversion";
logmsg "==================================";

@months = qw(Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec);
@weekDays = qw(Sun Mon Tue Wed Thu Fri Sat Sun);
my $starttime = time();
($second, $minute, $hour, $dayOfMonth, $month, $yearOffset, $dayOfWeek, $dayOfYear, $daylightSavings) = localtime($starttime);
$year = 1900 + $yearOffset;
$hour = sprintf("%02d", $hour);
$minute = sprintf("%02d", $minute);
$second = sprintf("%02d", $second);
$theTime = "$hour:$minute:$second, $weekDays[$dayOfWeek] $months[$month] $dayOfMonth, $year";
logmsg "Starting conversion at $theTime";

my $src_backslashed = $source_dir;
my $dst_backslashed = $dest_dir;
$src_backslashed =~ s/\//\\/g;
$dst_backslashed =~ s/\//\\/g;
logmsg "Source directory: ".$src_backslashed;
logmsg "Target directory: ".$dst_backslashed;

logmsg "Converting all ".uc($source_ext)." files to ".$dest_codec;
logmsg "Codec CLI options: ".$codec_options;
logmsg "DSP effects: ".$dsp_effects;
logmsg "dBpoweramp options: ".$dmc_options;

logmsg "------------------";


*blooper*
*blooper* Set up fork manager for parallel processing and 
*blooper* define number of processes to spawn for conversion
*blooper* (set to, e.g., number of CPUs. Use 0 to disable for testing)
*blooper*
my $pm = new Parallel::ForkManager($number_of_cpus);

*blooper* Callback for when a conversion child thread finishes
$pm->run_on_finish(
  sub { my ($pid, $exit_code, $ident) = @_;
    $dst = $ident;
    $dst =~ s/$source_dir(.*)\.$source_ext/$dest_dir$1\.$dest_ext/i;
    my $info = get_mp4info($dst);   *blooper* if converting to MP4, this will allow the encoding speed vs real-time to be calculated
    $total_audio_time += $info->{SECS};
    $total_converted_files += 1;
    $ident =~ s/$source_dir\/(.*)\.$source_ext/$1\.$dest_ext/;
    $ident =~ s/\//\\/g;
    *blooper*logmsg "    ...Finished $ident";
  }
);

*blooper* Callback for when a conversion child thread is initiated
$pm->run_on_start(
  sub { my ($pid,$ident)=@_;
    $ident =~ s/$source_dir\///;
    $ident =~ s/\//\\/g;
    logmsg "Converting $ident...";
  }
);


*blooper*
*blooper* Process files
*blooper*
$total_audio_time = 0;
$total_converted_files = 0;
foreach my $srcfile (@srcfiles) {
  my $destfile = $srcfile;
  $destfile =~ s/$source_dir(.*)\.$source_ext/$dest_dir$1\.$dest_ext/i;
  if (! -e $destfile) {
    $pm->start($srcfile) and next; *blooper* do the fork 
      $srcfile =~ s/\//\\/g;
      $destfile =~ s/\//\\/g;
      system('"'.$coreconverter_exe.'"' .
              ' -infile="' . $srcfile . '" ' .
              ' -outfile="' . $destfile . '" ' .
              $dmc_options .
              $dsp_effects .
              ' -convert_to="' . $dest_codec . '" ' .
              $codec_options);
    $pm->finish; *blooper* do the exit in the child process
  }
}

$pm->wait_all_children;

my $endtime = time();
($second, $minute, $hour, $dayOfMonth, $month, $yearOffset, $dayOfWeek, $dayOfYear, $daylightSavings) = localtime($endtime);
$year = 1900 + $yearOffset;
$hour = sprintf("%02d", $hour);
$minute = sprintf("%02d", $minute);
$second = sprintf("%02d", $second);
$theTime = "$hour:$minute:$second, $weekDays[$dayOfWeek] $months[$month] $dayOfMonth, $year";

$seconds = $endtime - $starttime;
@parts = gmtime($seconds);
$duration = sprintf ("%02d:%02d:%02d", @parts[2,1,0]);
if ($seconds > 0 && $total_audio_time > 0) {
  $realtime = sprintf("%.1f", $total_audio_time / $seconds);
} else {
  $realtime = 0;
}

@parts = gmtime($total_audio_time);
if ($total_audio_time > 0) {
  $playing_time = sprintf("%dd %dh %dm", @parts[7,2,1]);
} else {
  $playing_time = 0;
}

logmsg "------------------";
logmsg "Finished converting at $theTime.";
logmsg "Converted $total_converted_files files (total playing time $playing_time).";
logmsg "Conversion completed in $duration at ${realtime}x real-time encoding speed.\n";

*blooper* Close the logfile (if it was open)
close LOGFILE;

P.S. Why are the hashes that mark comments in my script appearing as '*blooper*' ?

[Spoon]

Re: Recursive Directory Scripting/Conversion

hash is disabled on the forum as spammers use it to hide spam messages (such as setting the font color to light blue).