This topic was archived here:

https://web.archive.org/web/20230000000000fw_/https://www.zenhax.com/viewtopic.php?t=14823

CMNDuser, posted Wed Feb 03, 2021 5:14 pm (61988)

---

Hello
I want to extract the tintin.bf file, as I think it contains the models from the player and enemies etc.
(This would make sense as the other 2 Bigfiles are the audio and the cutscenes)

I am however unable to do this using the tintin.bms script, as it gives an error code.

Does anyone have an idea if it would be possible to extract these files?
Thanks in advance

Error code:

Code:

  offset   filesize   filename
--------------------------------------
  004f6014 563264     84024479.bin

Error: the compressed LZO input is wrong or incomplete (-6)
Info:  algorithm   8
       offset      004f6014
       input size  0x0006abca 437194
       output size 0x00089840 563264
       result      0xffffffff -1

Error: the uncompressed data (-1) is bigger than the allocated buffer (563264)
       It usually means that data is not compressed or uses another algorithm

Last script line before the error or that produced the error:
  73  clog NAME OFFSET ZSIZE SIZE

tintin.bms

aluigi, posted Wed Feb 17, 2021 10:27 pm (62338)

---

Do you think you can upload the first 10 megabytes of that file?
I only need that portion of data for my analysis.

CMNDuser, posted Tue Feb 23, 2021 1:34 pm (62452)

---

aluigi wrote:

Do you think you can upload the first 10 megabytes of that file?
I only need that portion of data for my analysis.

I sadly don't know how to cut pieces out of the file, so here is the entire thing
https://drive.google.com/file/d/1Gp5ZTNDYzxfrHK9O7xQjl1AU7BOBCvda/view?usp=sharing

CMNDuser, posted Thu Mar 11, 2021 4:15 pm (62795)

---

Just wondering, did you see my response?

Savage, posted Mon Mar 22, 2021 8:01 pm (62973)

---

"The file is too big, maximum allowed size is 5 MiB."

Here is the sample of the file TINTIN.bf (the full file is 2,397,832,629 bytes), 15 MB, i hope it helps

https://krakenfiles.com/view/b01d0e4c94/file.html

aluigi, posted Fri Mar 26, 2021 12:16 pm (63045)

---

It seems like it uses chunked files and there is a sort of lzma header but any lzma decompression fails.
I don't have any solution at the moment.

tbmq008, posted Mon Dec 13, 2021 5:42 pm (68221)

---

OK, i have a bit more details to share now.
so it uses chunked files. and they don't seem to be using some sort of lzma compression stuff at all.

oh never mind it IS lzma, i'll share more details whenever possible.

OK, i just found out it uses lzma86 with a tweaked header. sorta like lzma86dechead or something. like this.

Code:

00 5D 00 00 40 00 7C 00 00 00 00 00 00 00 00 00 68 00 E5
^ blank byte
   ^ lzma signature
            ^ compression level 6
                  ^ decompressed chunk_size (64-bit var)
                                             ^ raw data i guess

so yes it is 100% lzma86dechead.

aluigi, posted Wed Dec 15, 2021 10:44 am (68254)

---

In my previous post it's written that any lzma decompression failed, I'm quite sure I correctly tested it because I have experience with the various lzma formats.
So if lzma_dynamic running on various positions of that input data failed, it means it's not real lzma.

tbmq008, posted Wed Dec 15, 2021 3:18 pm (68261)

---

i had to take a look at the game executable to see what kind of compression was it actually using for most of its files. i was sure it was using more than lzo1x but i had to dig deeper to find out.

that took me a long time than expected, i was looking at the functions that i believed would do the actual decompression work but turns out it was just memory handling. then i turned to the leftover Wii exe (LynWiiTintin.elf) and the funcion names were already there so i had little to no difficulty in locating the decompression functions at all.

so i pinned down a few functions (most of which had a "Lzma" name on them, the primary function for handling lzma compression is named "Lzma86_Decode" on the Wii executable i just mentioned previously) that did the actual decompression work, and compared the Wii exe to the exe i was previously looking at when it came to these functions. turns out they're almost identical to each other.

by the time i did i already decided it was using lzma86.

i can provide some files if you want.

aluigi, posted Wed Dec 15, 2021 5:18 pm (68266)

---

Maybe the samples provided in March were corrupted in some way?

If you have another sample, sure I can check it.
You can also try the lzma_dynamic decompression by yourself on the raw data dumped in a file:

Code:

math OFFSET = 0 # set your desired position in the file
comtype lzma_dynamic
get SIZE asize
math SIZE - OFFSET
clog "dump.dat" OFFSET SIZE SIZE

You can also set the desired offset in case you want to try different positions in the raw data.
The lzma algorithms in quickbms also support some settings in case the game uses a raw headerless lzma data (in quickbms it's called LZMA_0), in this case the only setting is the dictionary size, for example: comtype LZMA_0 0x400000

tbmq008, posted Thu Dec 16, 2021 1:34 am (68279)

---

alright. here are the first 32MB of TINTIN.bf from the Wii version of the Tintin game.
https://krakenfiles.com/view/xjEwNoHKHT/file.html

some of the compressed files coming from that bigfile is already available right here.
https://krakenfiles.com/view/CGZvh907Br/file.html

aluigi, posted Thu Dec 16, 2021 11:56 am (68288)

---

Ok I have updated the script to version 0.2.
I also checked some of the few old samples that were still available and it works with them too so I hope it has not lost any compatibility.
My only doubt is about a specific field which was used in the old script when TEST is 4 but it wasn't available in a sample I checked.

If someone can test this new script with other games I would be happy to know the results, hopefully better than the old one (more coverage?).

tbmq008, posted Thu Dec 16, 2021 1:55 pm (68292)

---

i can test the script. you can count on me.

---

well, something happened.

Code:

  00000000345a3780 380        VarDescMgr.dat
  00000000345a3840 115642     Univers.mdl

Error: the compressed LZMA input is wrong or incomplete (0)
Info:  algorithm   19
       offset      00000000345a3840
       input size  0x0000000000002fab 12203
       output size 0x000000000001c3ba 115642
       result      0xffffffffffffff9c -100

Error: there is an error with the decompression
       the returned output size is negative (-100)

Last script line before the error or that produced the error:
  79  clog NAME OFFSET ZSIZE SIZE

- OFFSET       0x00000000345a3840
- ZSIZE        0x0000000000002fab
- SIZE         0x000000000001c3ba
  coverage file 0    99%   874029544  878652420  . offset 00000000345a67eb

E:\quickbms>

the script tried to compress a lzma file that isn't actually such.
and that's just the tintin game, i don't know how the other games fare in this regard.

tbmq008, posted Thu Dec 16, 2021 4:07 pm (68297)

---

so, i tested another game with this script.
said game is the Wii version of Michael Jackson The Experience.

Code:

  00000000162bd270 63645628   Bad.bik
  00000000162bd710 9208778    Bad.wav
  00000000162bdbb0 0          Bad.wog
  00000000162bdc90 75010480   AnotherPartOfMe.bik
  00000000162be130 8658730    AnotherPartOfMe.wav
  00000000162be5d0 0          AnotherPartOfMe.wog
  00000000162be6b0 2213572    Credits.bik
  00000000162beb50 0          MainMenu.wog
  00000000162bec30 13750      Universe.var
  00000000162c0014 6168       Universe\trk\Run.fct
  00000000162c0ba8 6701       Universe\trk\Run.efc
  00000000162c1444 3307       Universe\Defines.h
  00000000162c19f4 55000      Universe.mdl
  00000000162c341c 22138      Math.fcg
  00000000162c4f9c 31278      Math.efg
  00000000162c732c 16065      Utilities.fcg
  00000000162c86fc 26318      Utilities.efg
  00000000162ca5c8 5126       ArrayFunctions.fcg
  00000000162caba4 14502      ArrayFunctions.efg
  00000000162cb8d8 27007      Messages.fcg
  00000000162ccdb4 43579      Messages.efg
  00000000162cf630 10596      Triiggers\Exec.fcg
  00000000162d00b8 16864      Triiggers\Exec.efg
  00000000162d0d94 6318       Triggers\Test.fcg
  00000000162d1584 7064       Triggers\Test.efg
  00000000162d1cb0 1966       Universe\Accessors.fcg
  00000000162d2110 3374       Universe\Accessors.efg
  00000000162d25e0 8208       Loading.fcg
  00000000162d3160 21203      Loading.efg
  00000000162d45f0 1662       MJ\Utilities.fcg
  00000000162d49e0 2560       MJ\Utilities.efg
  00000000162d4e60 0          Global Library.grp
  00000000162d4fd8 3152       LoadMaskDB.cfg
  00000000162d5644 6965       default.cfg
  0000000000000000 0          28017e73.bin
  00000000162d6380 262144     28017e73.bin

Error: the compressed LZO input is wrong or incomplete (-4)
Info:  algorithm   8
       offset      00000000162d6380
       input size  0x0000000000000001 1
       output size 0x0000000000040000 262144
       result      0xffffffffffffffff -1

Error: uncompressed data (-1) bigger than allocated buffer (262144)
       It usually means that data is not compressed or uses another algorithm

Last script line before the error or that produced the error:
  107 clog NAME OFFSET ZSIZE[x] CHUNK_SIZE

- OFFSET       0x00000000162d6380
- ZSIZE        0x0000000000000001
- SIZE         0x0000000000040000
  coverage file 0   100%!  3362242425 801267676  . offset 00000000162d6381

it's not great.

aluigi, posted Thu Dec 16, 2021 5:46 pm (68302)

---

In the first example the input and output size seem correct so I suppose that's a chunk not compressed with lzma.
Can you check it or provide a dump of the portion of data from offset 0x345a3840 of 0x2fab bytes?
We need also to check the value of the TEST field because it may be different than 6 in that chunk.
You can print that variable at runtime with this command at line 57 after it's read with Get: print "%TEST%"

In the second example it looks what I mentioned in my post, some archives have the TEST field 4 but some with and others without an additional field before CHUNKS.
Can you check this one too?

We only need to find a rule that works with all the samples.

tbmq008, posted Thu Dec 16, 2021 5:58 pm (68304)

---

on it.

tbmq008, posted Thu Dec 16, 2021 6:08 pm (68305)

---

i satisfied all that was requested above. except for the third request. here are the results.
there are some files with TEST 2, and one with TEST 0 (it's the one you asked me to provide a dump of).
all are included here. though i had only had to include 3 files here since i thought that's just about enough for what i had to include here.
https://krakenfiles.com/view/Asoi7Yh0ZR/file.html

some TEST 4 archives (RGH.bf, MJ.bf) work fine with the script to a point.

Code:

QuickBMS generic files extractor and reimporter 0.11.1 (64bit test)
by Luigi Auriemma
e-mail: [email protected]
web:    aluigi.org
        (Dec 15 2021 - 11:26:00)

                          quickbms.com  Homepage
                            zenhax.com  ZenHAX Forum
                     @zenhax @quickbms  Twitter & Scripts

- current_folder: E:\quickbms
- bms_folder:     E:\
- exe_folder:     E:\quickbms
- file_folder:    C:\[REDACTED]
- output_folder:  E:\quickbms
- temp_folder:    C:\Users\[REDACTED]\AppData\Local\Temp\
- open input file C:\[REDACTED]\RGH.BF
- open script E:\tintin.bms
- set output folder .

  offset           filesize   filename
--------------------------------------

Error: incomplete input file 0: C:\[REDACTED]\RGH.BF
       Can't read 370984 bytes from offset 0000000039e1feac.
       Anyway don't worry, it's possible that the BMS script has been written
       to exit in this way if it's reached the end of the archive so check it
       or contact its author or verify that all the files have been extracted.
       Please check the following coverage information to know if it's ok.

  coverage file 0    63%   619709061  971112108  . offset 0000000039e1feac

Last script line before the error or that produced the error:
  48  log NAME OFFSET SIZE

- OFFSET       0x0000000039d7a7d4
- SIZE         0x00000000002e24c6
  coverage file 0    63%   619709061  971112108  . offset 0000000039e1feac

Code:

QuickBMS generic files extractor and reimporter 0.11.1 (64bit test)
by Luigi Auriemma
e-mail: [email protected]
web:    aluigi.org
        (Dec 15 2021 - 11:26:00)

                          quickbms.com  Homepage
                            zenhax.com  ZenHAX Forum
                     @zenhax @quickbms  Twitter & Scripts

- current_folder: E:\quickbms
- bms_folder:     E:\
- exe_folder:     E:\quickbms
- file_folder:    C:\[REDACTED]
- output_folder:  E:\quickbms
- temp_folder:    C:\Users\[REDACTED]\AppData\Local\Temp\
- open input file C:\[REDACTED]\MJ.bf
- open script E:\tintin.bms
- set output folder .

  offset           filesize   filename
--------------------------------------

Error: the compressed LZO input is wrong or incomplete (-4)
Info:  algorithm   8
       offset      00000000162d6380
       input size  0x0000000000000001 1
       output size 0x0000000000040000 262144
       result      0xffffffffffffffff -1

Error: uncompressed data (-1) bigger than allocated buffer (262144)
       It usually means that data is not compressed or uses another algorithm

Last script line before the error or that produced the error:
  108 clog NAME OFFSET ZSIZE[x] CHUNK_SIZE

- OFFSET       0x00000000162d6380
- ZSIZE        0x0000000000000001
- SIZE         0x0000000000040000
  coverage file 0   100%!  3362242425 801267676  . offset 00000000162d6381

aluigi, posted Thu Jan 06, 2022 6:04 pm (68764)

---

Back to this topic but honestly I don't have any updates (also because I came here too late).
Anyway I don't know if 0x345a3840_0x2fab is compressed and how because most of the text is visible and the words aren't truncated.
While regarding the TEST field, it's a mess.
I think that it's more convenient for me to no longer touch that script, but obviously if someone provides a patch I will be happy to implement it