This topic was archived here:

https://web.archive.org/web/20230000000000fw_/https://www.zenhax.com/viewtopic.php?t=17460

sparr, posted Sun Sep 25, 2022 6:32 am (73575)

---

I have a script that works for my archive file in uncompressed form, but the archive itself is gzip'd. Is there some way I can wrap/prefix my script to handle that? The file is relatively small, so extracting it to memory or a temp file would be acceptable. I'd just prefer to do that within quickbms rather than having to add a layer of shell or other script around it.

sparr, posted Tue Sep 27, 2022 9:50 pm (73607)

---

Here's the script in question, which I'll post in its own thread if/when I get it fully working.

Code:

# quickbms script for the packages and pool files in the game Zero-K
# by Clarence "Sparr" Risher 

# packages/*.gz are index files, they must be gunzip'd before this script is applied
# pool/*/*.gz are the real files, named after their hashes, with their real names found in the index
# FIXME figure out how to gunzip the index file in this script instead

endian big
comtype gzip

get INDEXSZ asize 0

do
    get NAMESZ byte 0
    getdstring NAME NAMESZ 0

    # first byte of hash is used as a directory below
    get POOLDIR byte 0

    # 15 remaining bytes of hash, used as filename below
    # TODO find a way to do `getbits POOLHASH 120 0` then convert all 15 bytes to hex string
    get POOLHASH1 threebyte 0
    get POOLHASH2 long 0
    get POOLHASH3 long 0
    get POOLHASH4 long 0

    get UNK1 long 0 # 4 mystery bytes, samples: 465F7A73 2AFBD45C 43614B0F

    get FILESZ long 0

    # construct the path, which looks like "../pool/ff/fffffffffffffffffffffffffffff.gz"
    string POOLDIRX p "x" POOLDIR
    string POOLNAME1 p "x" POOLHASH1
    string POOLNAME2 p "x" POOLHASH2
    string POOLNAME3 p "x" POOLHASH3
    string POOLNAME4 p "x" POOLHASH4
    string POOLFILE = "../pool/"
    string POOLFILE   POOLDIRX
    string POOLFILE   "/"
    string POOLFILE   POOLNAME1
    string POOLFILE   POOLNAME2
    string POOLFILE   POOLNAME3
    string POOLFILE   POOLNAME4
    string POOLFILE   ".gz"

    open FDSE POOLFILE 1
    goto 0 1 # seek to beginning of file instead of remembering offset in previous file #1

    get ZSIZE asize 1
    clog NAME 0 ZSIZE FILESZ 1
    # FIXME filenames starting with dot have the dot stripped

    savepos OFFSET 0
while OFFSET < INDEXSZ # loop to the end of the file

spiritovod, posted Wed Sep 28, 2022 12:50 pm (73615)

---

Not sure what should be gzipped here, but if you want to do it explicitly (or if it doesn't work in all reimport modes), it can be done with:

Code:

comtype gzip_compress
get SIZE asize # for current file, or asize MEMORY_FILE for existing memory file
clog "file.gz" 0 SIZE SIZE # for current file, or SIZE MEMORY_FILE for existing memory file

Also, you can simplify some parts:

Code:

getdstring POOLHASH 15 # 0 is actually not required when you work with initially open file
... ...
string POOLHASH 0b POOLHASH
string POOLFILE p "../pool/x/%s.gz" POOLDIR POOLHASH
open FDSE POOLFILE 1

sparr, posted Wed Sep 28, 2022 4:37 pm (73618)

---

spiritovod wrote:

Not sure what should be gzipped here, but if you want to do it explicitly (or if it doesn't work in all reimport modes), it can be done with:

Code:

comtype gzip_compress
get SIZE asize # for current file, or asize MEMORY_FILE for existing memory file
clog "file.gz" 0 SIZE SIZE # for current file, or SIZE MEMORY_FILE for existing memory file

I think that would make the index itself appear as an extractable file, which I don't want but could probably tolerate. How do I then run the rest of my script on that decompressed file?

Thanks for the other improvement. It's so close, except `string ... 0b ...` makes upper case instead of lower case hex digits. Is that 0b functionality documented anywhere?

EDIT: aha, there's a "l" operator to lowercase a string!

spiritovod, posted Wed Sep 28, 2022 8:57 pm (73621)

---

That 0 prefix is explained in documentation for string command. By using it you can avoid certain issues while converting strings with some operators.

If you want to process those gzip files further in the same script you can use something like this

Code:

open FDSE POOLFILE 1
# in case if gzip files has custom header or additional data, you must process it first here
# otherwise you can just use the following:
get SIZE asize 1
clog MEMORY_FILE 0 SIZE SIZE 1

# now you can do things with unpacked to memory file
# get MAGIC long MEMORY_FILE // etc