title
Products            Buy            Support Forum            Professional            About            Codec Central
 
Page 1 of 2 12 LastLast
Results 1 to 15 of 19

Thread: CoreConverter and very large files

  1. #1

    CoreConverter and very large files

    Using ref 12.1 on a machine with lots of muscle.

    Got some whole album rips in wma lossless and converting them to flac is causing me some real trouble. I've converted tens of thousands of <= 200MB with no trouble. But around 200MB, something seems to go wrong. There's also some substantial memory allocation > 2GB while converting.

    I'm looking into the issue, and I figured I would ask if anyone is aware of a problem with large files and coreconverter under tight physical memory conditions. The memory allocation does not seem abnormal as it grows size-proportional to the file you are converting, but I make a note of it since there's only 2GB of physical RAM in the machine. virtual limits out at around 6GB.

    I notice that around 4GB of peak commit charge, the files just seem to stop converting, with no error notice. The progress bar does not get to 100%, and there are no tags in the "converted" files.

    Richard

  2. #2
    dBpoweramp Guru LtData's Avatar
    Join Date
    May 2004
    Location
    USA
    Posts
    8,288

    Re: CoreConverter and very large files

    4GB is the limit that a 32-bit program can address. Your 200MB files, what format are they? If you look at the "Audio Properties" what size does it say the file is originally aka uncompressed?

  3. #3

    Re: CoreConverter and very large files

    Below is from the properties page.. when I try and convert this file using GUI interface, it says finished all of a sudden halfway through the conversion. At the point where it hits 2GB of commit charge. It leaves the incomplete file (204,111KB) on the hard drive and reports no error. Using a signed int as a ptr?

    Artist: Dimitri from Paris
    Title: Mixed Dance
    Album: House Music All Night Long
    Track: 1
    Disc:
    Genre: Dance
    Year: 2005
    Rating:
    Composer:
    Size: 530.59 MB (68% of original, 1.4 to 1 compression)
    Original Size: 772.89 MB
    Length: 1 hour 16 minutes
    Channels: 2 (stereo)
    Sample Rate: 44.1 KHz;
    Sample Size: 16 bit
    Bit Rate: 1,411 kbps
    Encoder: Windows Media Audio 9.1 Lossless
    Encoder Settings: VBR Quality 100, 44 kHz, 2 channel 16 bit 1-pass VBR
    Audio Quality: Perfect (Lossless)
    Play Count:
    Last Played:
    Contains: ID Tag [WMA ID Tag]
    Channel Mapping:
    File: Dimitri from Paris - House Mixed
    Type: Windows Media Audio file [.wma]

  4. #4
    dBpoweramp Guru LtData's Avatar
    Join Date
    May 2004
    Location
    USA
    Posts
    8,288

    Re: CoreConverter and very large files

    What format are you trying to convert to? What settings are you using? Are you running any DSP effects?

  5. #5

    Re: CoreConverter and very large files

    Windows server 2003
    Converting to FLAC
    Compression Level 5
    After encoding verify written audio [x]
    Output location: Original Folder
    DSP Effects: None

    I upgraded from ref 12 to ref 12.1. I immediately attempted to do a conversion.

  6. #6
    Administrator
    Join Date
    Apr 2002
    Posts
    43,854

    Re: CoreConverter and very large files

    >2GB of commit charge. It leaves the incomplete file (204,111KB)

    Entirely down to windows, we do not handle allocations.

    Try converting the file to test conversion, set at limited to x4 encode speed, it should take the true length / 4 if dbpoweramp can read the file.

  7. #7

    Re: CoreConverter and very large files

    On winserver2003 Using test decoder at 4X, it fails (but reports success) at 6min22sec of 17min process and around 2.1GB RAM used.

    On XP (with 1GB RAM) using using test decoder at default, it succeeds in 2 min and around 1GB RAM used.

    Both systems using ref12.1 and wma pro 9.

    On the surface, seems to be an OS issue. But could also be a CPU issue since the XP machine uses P4 3.2GHz, and the 2k3 machine uses Xeon E5310 at 1.6GHz.

    But the Xeon machine also uses something called "Physical Address Translation" as reported by "system". And has 2GB RAM. I guess that's a hardware virtual memory manager. Possibly something going wrong in there?

  8. #8
    Administrator
    Join Date
    Apr 2002
    Posts
    43,854

    Re: CoreConverter and very large files

    If your virtual memory did not work then you would see windows crashing.

  9. #9

    Re: CoreConverter and very large files

    I used a boot.ini switch like: /MAXMEM=1024, and it still died at 2GB. A 32bit process can access 2GB of memory unless using the /3GB boot.ini switch. I tried the /3GB and /PAE switches, but I guess CoreConverter is not linked/compiled with /LARGEADDRESSAWARE. While windows 32bit total virtual address space (without /PAE switch) is 4GB our wall seems to be the 2GB per process barrier. It is still very strange though. Because XP converts the file without using so much memory.

    XP uses around 1.1GB virtual memory to complete 100% of the 530MB wma file, and 2k3 uses 2.1GB of virtual memory to complete 1/3 of the same file. Something is going wrong someplace.

    The E5310 is a Xeon chip, so I am considering redoing the OS install as 64bit. I have searched around, but there's not a definitive statement like: "ref 12.1 runs native 64bit on winxp 64bit, win server 2003 64bit." Is ref 12.1 compiled for 64 bit?

    Richard

  10. #10
    dBpoweramp Guru LtData's Avatar
    Join Date
    May 2004
    Location
    USA
    Posts
    8,288

    Re: CoreConverter and very large files

    No, dMC is only a 32-bit program.

    Can you convert your WMA lossless file to a WAV file?

  11. #11
    Administrator
    Join Date
    Apr 2002
    Posts
    43,854

    Re: CoreConverter and very large files

    Our program will not be allocating that memory (we convert in small chunks), rather the wma subsystem built into the operating system, or some kind of file caching.

  12. #12

    Re: CoreConverter and very large files

    Not sure what it is. But I guess the test converter really is a test "decoder". So I went to Microsoft and downloaded their encoder so I could see what their "decoder" would do:

    http://www.microsoft.com/windows/win...r/default.mspx

    This resulted in my 1 hour wma lossless track taking 8 min to convert to a wma 192 VBR (the default). The system used 180MB RAM total throughout the conversion. I guess that means something is going wrong even on XP where it is using 1GB RAM.

  13. #13
    Administrator
    Join Date
    Apr 2002
    Posts
    43,854

    Re: CoreConverter and very large files

    Actually there was a buffer leak, it is now been fixed, it should take no more than 5 mb now...

  14. #14

    Re: CoreConverter and very large files

    I'm very happy to hear that it is fixed!

    Is that fix included in the beta or in the 12.2 reference currently available through the update process?

  15. #15
    dBpoweramp Guru LtData's Avatar
    Join Date
    May 2004
    Location
    USA
    Posts
    8,288

    Re: CoreConverter and very large files

    The WMA codec with the fix has been released, use the "Check for Update" button in dBpoweramp Configuration to get it.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •