This topic was archived here:

https://web.archive.org/web/20230000000000fw_/https://www.zenhax.com/viewtopic.php?t=4953

AlphaTwentyThree, posted Thu Sep 14, 2017 12:37 pm (26498)

---

Hello there!

I'm currently ripping the music of PSX games and for this matter, I've written some scripts to help me do this. I will post the scripts I have here and update when I write new ones.

First of all, you'll need an ISO browser that lets you extract XA (audio) and STR (video) files. For this purpose I'm using IsoBuster (https://www.isobuster.com/) with the "extract RAW" option.

Second, here are two scripts that will process those two file types

1. unecm

http://www.theisozone.com/downloads/pla ... ols/unecm/
Tool to decompress ecm files often used in PSX images.


2. XA deinterleaver & splitter

Just as it sounds. Automatically extracts all the streams out of interleaved XA files. I've implemented various types of split and end markers to support plenty of XA variants. If you find one that doesn't work with this, please post here.
You can manually overwrite the layer count and prevent the splitting/silence cutting of the layers. The scripts also adds a CDXA header so you can listen to the result. This is mainly for developer reasons.
Practical: If you set LAYERS to 1 and leave NOSPLIT at 0 you can add a CDXA header to a headerless CDXA stream. The silence at the end is automatically removed.

Code:

# deinterleaves and splits RAW-extracted XA files 
# rev 2017-11-02
# to playable XA files with CDXA header
# written by AlphaTwentyThree of Zenhax
# script for QuickBMS http://quickbms.aluigi.org

idstring \x00\xFF\xFF\xFF\xFF\xFF\xFF\xFF
callfunction get_layers 1
#set LAYERS 1
set NOSPLIT 0 # set this to 1 to prevent splitting of all layers
set VERBOSE 1
set SPLIT_BY_SILENCE 0

xmath BLOCKSIZE "0x930 * LAYERS"
get CYCLES asize
math CYCLES /= BLOCKSIZE
callfunction deinterleave

startfunction get_layers
   set OFFSET 0x11
   math OFFSET  = 0x930
   goto OFFSET
   get START byte # actually second one as 0 repeats
   math OFFSET  = 0x930
   xmath GO "START   1"
   for i = GO
      goto OFFSET
      get TEST byte
      if TEST == START
         break
      endif
      math OFFSET  = 0x930
   next i
   xmath LAYERS "i - START"
endfunction
   
startfunction deinterleave
   set CUT_SILENCE 1
   xmath PSIZE "CYCLES * 0x930"
   get BNAME basename
   goto 0x13
   get TEST byte
   #if TEST == 0
   #   set SPLIT_BY_SILENCE 1
   #endif
   for i = 1 <= LAYERS
      set SINGLE 1
      putVarChr MEMORY_FILE PSIZE 0
      log MEMORY_FILE 0 0
      xmath JUMPOFFSET "(i-1)*0x930"
      for k = 1 <= CYCLES
         goto JUMPOFFSET
         getDstring WDATA 0x930
         putDstring WDATA 0x930 MEMORY_FILE
         math JUMPOFFSET  = BLOCKSIZE
      next k
      if NOSPLIT == 1
         set SINGLE 1
         set SIZE PSIZE
      else
         set m 1 # 1 segment start
         callfunction get_segments 1
      endif
      if SINGLE == 1
         set OFFSET 0
         if SIZE != 0
            if NOSPLIT != 1
               if CUT_SILENCE != 0
                  callfunction cut_silence 1
               endif
            endif
            callfunction CDXA 1
            string NAME p= "%s_%i.xa" BNAME i
            get SIZE asize MEMORY_FILE2
            log NAME 0 SIZE MEMORY_FILE2
         endif
      else
         if VERBOSE == 1
            print "found %m% segments in layer %i%"
         endif
         set c 1
         for n = 1 <= m # walk through array with segment TOC
            getArray OFFSET 0 n
            getArray SIZE 1 n
            if SIZE != ""
               callfunction CDXA 1
               string NAME p= "%s_%i_%i.xa" BNAME i c
               get SIZE asize MEMORY_FILE2
               log NAME 0 SIZE MEMORY_FILE2
               math c  = 1
               putArray 0 n "" # IMPORTANT!!!
               putArray 1 n ""
            endif
         next n
      endif
   next i
endfunction

startfunction get_segments
   set SEARCH 0x10
   xmath LAYER_CYCLES "PSIZE / 0x930"
   set ACTIVE 1 # to determine if a new start is needed
   set OFFSET 0
   set SILENCE_CUT 0
   for l = 1 <= LAYER_CYCLES
      if SPLIT_BY_SILENCE == 1
         callfunction splitbysilence 1
         break
      endif
      goto SEARCH MEMORY_FILE
      get TEST4 byte MEMORY_FILE # stream ident code
      get LAYERNUMBER byte MEMORY_FILE
      if l == 1 # set first ident to identify next one
         set IDENT TEST4
      endif
      get TEST byte MEMORY_FILE # audio identifier, usually 0x64
      get TEST2 byte MEMORY_FILE # normally 1 or 4, only 0 when silence-split
      get DUMMY long MEMORY_FILE
      get TEST3 byte MEMORY_FILE # needs to be 0x0c
      if ACTIVE == 1 # offset is set, search for new start
         if TEST == 0xe4 || TEST == 0x80 || TEST == 0 || TEST4 == 0xff || TEST == 0x60 # stream end marker
            if TEST == 0xe4
               math SEARCH  = 0x930
               math l  = 1
            endif
            if VERBOSE == 1
               print "split by end marker"
            endif
            callfunction enter 1
            set ACTIVE 0
            set CUT_SILENCE 0
         elif TEST == 0x48 # stream cut, stays active
            if VERBOSE == 1
               print "split by 0x48 stream cut"
            endif
            set SILENCE_CUT 1
            callfunction enter 1
            if SIZE > 0x930
               math m  = 1
            endif
            xmath OFFSET "SEARCH - 0x10"
            set SINGLE 0
         elif TEST == 0xc2 # do nothing, jump one forward
            if VERBOSE == 1
               print "split by inactive block"
            endif
            set ACTIVE 0
         elif TEST4 != IDENT # stays active
            if VERBOSE == 1
               print "split by segment identifier"
            endif
            callfunction enter 1
            set SINGLE 0
            set IDENT TEST4 # set new ident
            math m  = 1
            set CUT_SILENCE 0
            xmath OFFSET "SEARCH - 0x10"
         elif TEST4 == 0x7f # dummy silence
            if VERBOSE == 1
               print "split by dummy silence"
            endif
            set ACTIVE 0
            set CUT_SILENCE 0
         endif
      elif ACTIVE == 0 # search new start
         if TEST == 0x64 && TEST2 != 0 && TEST3 != 0
            math m  = 1
            xmath OFFSET "SEARCH - 0x10"
            set SINGLE 0 # second stream found
            set ACTIVE 1
         elif TEST == 0 && TEST2 == 0 # rest of layer is empty
            break
         endif
      endif
      math SEARCH  = 0x930
   next l
   if SILENCE_CUT == 1
      math m -= 1
   endif
   
endfunction

startfunction splitbysilence
   set SEARCH 0x898 # test for 0x0c0c0c0c...
   xmath LAYER_CYCLES "PSIZE / 0x930"
   set OFFSET 0
   get FSIZE asize MEMORY_FILE
   for l = 1 <= LAYER_CYCLES
      goto SEARCH MEMORY_FILE
      get TEST longlong MEMORY_FILE
      if TEST == 0x0c0c0c0c0c0c0c0c
         math SEARCH  = 0x90 # last sample of block
         goto SEARCH MEMORY_FILE
         get TEST long MEMORY_FILE
         if TEST == 0
            math SEARCH  = 0x18
            if VERBOSE == 1
               print "split by 0x|:0c:| silence"
            endif
            callfunction enter 1
            if SIZE > 0x930
               math m  = 1
            endif
            xmath OFFSET "SEARCH - 0x10"
            math SEARCH  = 0x888
            set SINGLE 0
         endif
      endif
      math SEARCH  = 0x930
      if SEARCH >= FSIZE
         break
      endif
   next l
   get SEARCH asize MEMORY_FILE # last segment
   math SEARCH  = 0x10
   callfunction enter 1
endfunction

startfunction enter
   math SEARCH -= 0x10 # set to start of next block 
   xmath SIZE "SEARCH - OFFSET"
   if SIZE > 0x930 # sometimes there's a 0x48-only marker loop at the end
      #print "SEG %m%: %SIZE% bytes @ offset %OFFSET%"
      putArray 0 m OFFSET
      putArray 1 m SIZE
   else 
      # skip enter - also works for end silences when split by silence
      putArray 0 m ""
      putArray 1 m ""
      math SEARCH  = 0x930 # skip one block
      math l  = 1          # <--- don't forget this!
   endif
   math SEARCH  = 0x10
endfunction

startfunction cut_silence
   xmath SILENCE_CYCLES "PSIZE / 0x930"
   xmath STARTBLOCK "SILENCE_CYCLES / 5" # estimate
   xmath SIZE "STARTBLOCK * 0x930   0x28"
   for o = STARTBLOCK <= SILENCE_CYCLES
      goto SIZE MEMORY_FILE
      get TEST longlong MEMORY_FILE
      if TEST == 0
         break
      endif
      math SIZE  = 0x930
   next o
   math SIZE -= 0x28
   if o == SILENCE_CYCLES
      math SIZE  = 0x930
   endif
endfunction

startfunction CDXA
   xmath PSIZE2 "SIZE   0x2c"
   putVarChr MEMORY_FILE2 PSIZE2 0
   log MEMORY_FILE2 0 0
   set MEMORY_FILE2 binary "\x52\x49\x46\x46\xe4\x04\xbe\x02\x43\x44\x58\x41\x66\x6d\x74\x20\x10\x00\x00\x00\x00\x00\x00\x00\x01\x55\x58\x41\x01\x00\x00\x00\x00\x00\x00\x00\x64\x61\x74\x61\xc0\x04\xbe\x02"
   xmath RIFFSIZE "SIZE   0x24"
   putVarChr MEMORY_FILE2 0x04 RIFFSIZE long
   putVarChr MEMORY_FILE2 0x28 SIZE long
   append
   log MEMORY_FILE2 OFFSET SIZE MEMORY_FILE
   append
endfunction

3. STR audio extractor

Does what it says: extract the XA stream from an STR movie file, playable with winamp/foobar and the vgmstream plugin. At the moment, only single streams are supported.

Code:

# extracts CDXA audio from Playstation STR movie files
# rev 2017-09-14
# written by AlphaTwentyThree of Zenhax
# script for QuickBMS http://quickbms.aluigi.org

set MEMORY_FILE binary "\x52\x49\x46\x46\xe4\x04\xbe\x02\x43\x44\x58\x41\x66\x6d\x74\x20\x10\x0\x0\x0\x0\x0\x0\x0\x1\x55\x58\x41\x1\x0\x0\x0\x0\x0\x0\x0\x64\x61\x74\x61\xc0\x4\xbe\x2"
get FSIZE asize
set OFFSET 0
append
DO
   xmath IDENT "OFFSET   0x12"
   goto IDENT
   get DAT byte
   if DAT == 0x64 # audio marker
      log MEMORY_FILE OFFSET 0x930
   elif DAT == 0xe4
      log MEMORY_FILE OFFSET 0x930
      break
   endif
   math OFFSET  = 0x930
WHILE OFFSET < FSIZE
append
get SIZE asize MEMORY_FILE
set RIFFSIZE SIZE
math RIFFSIZE -= 8
set DSIZE SIZE
math DSIZE -= 0x2c
putVarChr MEMORY_FILE 0x04 RIFFSIZE long
putVarChr MEMORY_FILE 0x28 DSIZE long
get NAME basename
string NAME  = ".xa"
log NAME 0 SIZE MEMORY_FILE

4. PSound by snailrush

http://snailrush.online.fr/
This little program lets you scan archives for VAG and SS2 files to check if you'll need to extract the contents.


5. PSF by Mark Grass

http://www.romhacking.net/utilities/1020/
Tool to convert SEQ/VH/VB triplets to playable PSF files.


That's it for the moment. I will update this thread with scripts for individual games.

AlphaTwentyThree, posted Tue Sep 19, 2017 10:58 pm (26627)

---

Updated the de-interleave script to support yet two more variants.

AnonBaiter, posted Mon Sep 25, 2017 10:55 pm (26839)

---

Your STR audio script forgot this:

Code:

   if DAT == 0x64 # audio marker
      log MEMORY_FILE OFFSET 0x930
   elif DAT == 0xe4 # "end-of-audio" marker
      log MEMORY_FILE OFFSET 0x930
      break
   endif

In other words, there's no need to avoid the last XA audio sample.

AlphaTwentyThree, posted Tue Sep 26, 2017 5:46 am (26846)

---

Ok, now I need to re-rip all PSX games done so far because the last audio sample is missing *facepalm*

AlphaTwentyThree, posted Sat Oct 21, 2017 9:40 pm (27574)

---

Uhm, can somebody tell me where the "edit" function went?

AnonBaiter, posted Sat Oct 21, 2017 10:37 pm (27576)

---

AlphaTwentyThree wrote:

Uhm, can somebody tell me where the "edit" function went?

https://zenhax.com/viewtopic.php?f=9&t=4012&start=20#p27512
Scroll down from here.

AlphaTwentyThree, posted Tue Oct 31, 2017 10:13 am (29576)

---

THANKS FOR BRINGING BACK THE EDIT FUNCTION!!!

Updated deinterleave script to support another two variants!

aluigi, posted Tue Oct 31, 2017 10:18 am (29579)

---

Sorry again for that
phpbb sux badly in these situations tbh

AlphaTwentyThree, posted Wed Nov 01, 2017 10:14 am (29610)

---

UPDATE

Updated deinterleave script once again.
DON'T USE VERSION 2017-10-31 anymore, it had a crude error!

AlphaTwentyThree, posted Thu Nov 02, 2017 7:48 pm (29668)

---

UPDATE

Another deinterleaver update. This time I encountered a file that would be automatically split by silence as an identifier that is normally 1 is 0 buuuuuuut in this case it was wrong so I had to make the SPLIT_BY_SILENCE a toggle. Leave it at 0 and only activate it when you have a file that isn't split properly.

AnonBaiter, posted Fri Dec 08, 2017 2:16 am (30571)

---

AlphaTwentyThree, as of this date vgmstream can already support CD-XA file detection. That means adding RIFF headers to .XA files are now completely unnecessary.

AlphaTwentyThree, posted Fri Dec 08, 2017 10:07 am (30579)

---

Ah, nice to know. Layer splitting is still needed I guess?
I'll leave the header inside the script - never change a running system.

AlphaTwentyThree, posted Fri Dec 08, 2017 10:10 am (30580)

---

Btw, I will speed up the audio extraction script by severak 100% some day.

AnonBaiter, posted Fri Dec 08, 2017 3:30 pm (30585)

---

AlphaTwentyThree wrote:

Layer splitting is still needed I guess?

Of course. Right now vgmstream will play some XA file that contains a bunch of "layers" as a single file, that is if the CD sectors are even included in the first place.

AlphaTwentyThree wrote:

I'll leave the header inside the script - never change a running system.

To be honest though, lately I just disliked seeing some quickBMS script pop up that "handles" these "playable files" out of some strange format that these vgmstream programmers(such as bnnm, which makes the most changes into it as of now) could otherwise handle at any moment.

Because of the constant updates vgmstream ended up receiving, I was pretty much discouraged to update my script that could handle SGXD files that contained this "AT9" audio codec thing.

If anything, let me just give you my piece of advice though: if you really want playable files on vgmstream, I suggest you talk about these files first with whoever's programming it at the moment at the hcs64 message boards before you head any further.