aluigi, posted Sun Nov 01, 2015 2:36 pm (9170)

---

An update about the tests I'm performing and my notes.
Regarding the FLAG field, I have found that &4 is used for that compression that doesn't work where there is a 32bit ZSIZE field at the beginning of the compressed data.
In my opinion it's something like a "solid" archive or similar.
In fact the first file can be decompressed correctly but all the others fail.

Some of these archives are EngineShaders and LowText.

All the others (current and old ones) seem to work perfectly.

The current script is now available on my website:
http://aluigi.org/bms/lspk.bms

aluigi, posted Sun Nov 01, 2015 3:35 pm (9175)

---

Just finished a test by using LZ4_decompress_safe_continue which is meant to perform a sort of continuous decompression.
Still the same results: first file ok but the others are partially invalid (for example 0x44 0x80 0x40 header instead of DDS and so on).

spider91, posted Thu Nov 05, 2015 12:53 pm (9332)

---

Any progress with that files?

aluigi, posted Fri Nov 06, 2015 7:10 am (9351)

---

From my side there will be no updates at the moment, I guess.

markfiter, posted Sun Nov 08, 2015 9:11 am (9430)

---

aluigi, are you still trying to do decompressor\compressor for all files?

aluigi, posted Sun Nov 08, 2015 10:44 am (9433)

---

Currently there is no solution for those files extracted in the UNSUPPORTED folder.
If and when there will be updates, they will be posted here.

spider91, posted Wed Nov 11, 2015 11:10 am (9599)

---

LowTex.pak can be extracted now, don't know if thats the best way, but .dds seems to be valid. Also Sound.pak and Textures.pak can be extracted too (there was an error in script).

Code:

get PAK_NAME basename
get PAK_EXT extension

math FLAGS = 0
math BASE_OFF = 0

getdstring SIGN 4
goto -4
getdstring SIGN2 4
goto 0

if SIGN2 == "LSPK"

    comtype lz4
    goto -0x28
    get VER long
    get INFO_OFF long
    get INFO_SIZE long
    get PAKS short
    get FLAGS short
    getdstring HASH 16
    get DUMMY long
    idstring "LSPK"
    goto INFO_OFF
    get FILES long
    savepos OFFSET
    math INFO_SIZE - 4

    if TMP > INFO_SIZE
        xmath TMP "FILES * 280"
        clog MEMORY_FILE OFFSET INFO_SIZE TMP
    else
        log MEMORY_FILE OFFSET INFO_SIZE
    endif

else

    if SIGN == "LSPK"
        idstring "LSPK"
    endif
    get VER long
    get BASE_OFF long
    if VER <= 9
        get PAKS short
        get FLAGS short 
        get INFO_SIZE long
        get ZERO byte
    else
        get INFO_SIZE long
        get PAKS short
        get FLAGS short 
    endif
    get FILES long

    savepos OFFSET
    log MEMORY_FILE OFFSET INFO_SIZE

endif

math CURRENT_PAK = 0
math LAST_OFFSET = 0

if FLAGS & 4
    comtype lz4f
    math SIZE = INFO_OFF
    math SIZE *= 10
    clog MEMORY_FILE2 0 INFO_OFF SIZE
    math SOLID_OFF = 0
endif

for i = 0 < FILES
    getdstring NAME 256 MEMORY_FILE
    get OFFSET long MEMORY_FILE
    get ZSIZE long MEMORY_FILE
    get SIZE long MEMORY_FILE
    get PAK_NUM long MEMORY_FILE
    if VER >= 10
        get ZIP long MEMORY_FILE
        get CRC long MEMORY_FILE

        math ZIP & 0xf
        if ZIP == 0
            comtype copy
        elif ZIP == 1
            comtype zlib
        elif ZIP == 2
            comtype lz4
        else
            print "Error: unknown compression type %ZIP%"
            cleanexit
        endif
    endif

    if PAK_NUM != CURRENT_PAK
        get TMP asize
        math LAST_OFFSET   TMP

        if PAK_NUM == 0
            string TMP p "%s.%s"    PAK_NAME         PAK_EXT
        else
            string TMP p "%s_%d.%s" PAK_NAME PAK_NUM PAK_EXT
        endif
        open FDSE TMP
        math CURRENT_PAK = PAK_NUM
        if PAK_NUM > 0
            math BASE_OFF = 0
        endif
    endif

    math OFFSET   BASE_OFF

    if FLAGS & 4
        log NAME SOLID_OFF SIZE MEMORY_FILE2
        math SOLID_OFF  = SIZE
    else
        if SIZE == 0
            log NAME OFFSET ZSIZE
        else
            clog NAME OFFSET ZSIZE SIZE
        endif
    endif
next i

aluigi, posted Wed Nov 11, 2015 11:31 am (9601)

---

So, basically, it's just a solid archive as I said but instead of trying complex things like parsing chunks and so on, I had simply to launch lz4 (lz4f to be exact) on the whole archive...
Excellent job spider91, I have updated the script to 0.3.
Thanks.

Just a note, I have seen you removed "math OFFSET - LAST_OFFSET", is that a mistake?
That operation is necessary to get the correct offset for the multi-part archives.

I have checked if the uncompressed size of the solid archives is written somewhere but no way, so *10 is the only solution at the moment.

spider91, posted Wed Nov 11, 2015 11:58 am (9603)

---

Quote:

Just a note, I have seen you removed "math OFFSET - LAST_OFFSET", is that a mistake?

Yep, it is. It gives an error when script goes to sound_1.pak or textures_1.pak. It was something about 0 - LAST_OFFSET, so it caused too big number and script crashed at that moment.

Quote:

I have checked if the uncompressed size of the solid archives is written somewhere but no way, so *10 is the only solution at the moment.

I think it can be calculated by mathing SIZE for every file in TOC, but i'm too lazy to do this.

spider91, posted Wed Nov 11, 2015 12:11 pm (9604)

---

Here it is http://puu.sh/lhxOc/a1cf1a8945.png . Same thing for Textures_1.pak.

aluigi, posted Wed Nov 11, 2015 2:19 pm (9613)

---

Ah ok, I didn't have the whole files so I guess the LAST_OFFSET thing was just a misunderstanding in the first stage of my analysis.
Script 0.3.1