This topic was archived here:

https://web.archive.org/web/20230000000000fw_/https://www.zenhax.com/viewtopic.php?t=380

Mygoshi, posted Sat Nov 15, 2014 8:04 pm (1795)

---

Hey everyone,

I'm trying to extract the *.FIB archive from LEGO Star Wars II (Nintendo DS version).

To my knowledge there are no ways to do this ATM so I felt like I could look for help here.

Samples: (expired)

Thanks in advance!

Mygoshi, posted Sat Nov 15, 2014 8:15 pm (1796)

---

To add more information to my request, I noticed most of the DS games use a SoundData.sdat and in these cases it's easy to output playable music, not for this case.

Mygoshi, posted Sun Nov 16, 2014 8:56 pm (1825)

---

My method (for the music) is to use VGMToolbox, and I simply drag and drop the NDS rom into the window which will produce automatically for me playable *.2SF files, decoded with 2SF decoder.

Mygoshi, posted Wed Nov 19, 2014 5:14 pm (1870)

---

Now this case is a little bit strange because I've never seen something like StarWars.fib, and even if what it looks like is not important, even the content of it seems very different.

aluigi, posted Wed Nov 19, 2014 5:38 pm (1871)

---

I already checked it and I pass.

m64m, posted Thu Dec 04, 2014 9:00 pm (2103)

---

Aluigi, would you be willing to give the format another try? I've found two titles that have a .CSV file alongside the .FIB, which contains the path and some extra information for each file within the .FIB archive. Here is the .CSV file from Harry Potter Years 5-7 (3DS): http://puu.sh/dh7Yi/1f93a0e282.csv

If you're up for it, I'll PM you a link to the FIB.

aluigi, posted Sun Dec 07, 2014 8:57 am (2120)

---

Sorry but I don't get the format of this archive.
I suspect it's obfuscated.

m64m, posted Sun Dec 07, 2014 10:16 pm (2126)

---

Ok, thanks for trying!

FireFly, posted Mon Aug 03, 2015 8:47 am (6765)

---

I have been working on reversing this format (FUSE) recently for another person, and although I seem to have figured out the compression, I still can't piece it all together and get data that I expect out of it.

The overall structure of FUSE is: there's a header { char magic[8] = "FUSE1.00"; u32 num_files; u32 zero1 = 0; u32 filesize; }. This is followed by compressed data for the first file, followed by compressed data for the second file, etc for num_files files.

Each compressed data starts with u32 compressed_size | 0x40000000, followed by the actual data. The compression is very similar to LZ. Triples (to_copy, then_copy, offset) are encoded in a variable-length way:

Code:

    Bytes                                 to_copy  then_copy            offset
  -----------------------------------------------------------------------------
    0yyyxxzz zzzzzzzz                      x        y   3                z   1
    10yyyyyy xxzzzzzz zzzzzzzz             x        y   4                z   1
    110xxyyy zzzzzzzz zzzzzzzz yyyyyyyy    x        (y1 << 7)   y2   5   z   1
    111xxxxx                               (x 1)*4  0                    0
    (0xFD)                                 3        0                    0
    (0xFE)                                 2        0                    0
    (0xFF)                                 1        0                    0

Decompressing this way seems to work perfectly with text files in the archives I've tried (using the accompanying CSV to derive filenames, offset and decompressed filesize). However, as I mentioned I've stumbled upon some issues.

Some files seem to not be compressed at all, but instead just stored verbatim. The weird thing is that in this case there doesn't seem to be any reasonable indicator of this, either in the CSV or the actual FUSE archive, and the "compressed" filesize (which had a 0x40 high byte in the compressed case) seems to be completely absent. Otherwise, if the filesize was still present I'd think that the high bits just act as a bitfield to store info such as whether the file is compressed or not. Non-compressed files include very big files with quite a lot of repetition, so it seems like compression would've made sense. It also includes very small files, and some in-between. There's relatively few uncompressed files.

Next is the main problem I've been stumped by: although text files decompress just fine, many kinds of binary files (png and tga images as well as xls spreadsheets, judging by filenames) seem to look nothing like what they're supposed to do. Instead, all seem to have a common header format. However, the rest of the data looks structured/not compressed or encrypted at all, so I'm not sure what's going on here. In the case of xls, several strings are visible in plaintext.

Mygoshi, posted Mon Aug 03, 2015 9:22 am (6766)

---

Thanks for your research on this.

FireFly, posted Tue Aug 04, 2015 11:36 am (6778)

---

Oh, you don't happen to know if the format is documented in detail anywhere? What tool did you use to extract the fib archives?

Mygoshi, posted Tue Aug 04, 2015 8:34 pm (6782)

---

I already extracted the fib files. The extracted files are instruments for the sequenced music. They're .bin files.

MrJPGames, posted Sat Oct 31, 2015 3:39 pm (9115)

---

Mygoshi wrote:

I already extracted the fib files. The extracted files are instruments/Sequenced musics. They're .bin files.

How did you do this? I'm trying to extract a fib file for Lego HP 5-7 for the 3DS which is probably the same as it also has the .cvs and .fib combo (and the games are pretty much the same anyway)

AlphaTwentyThree, posted Sat Aug 26, 2017 9:22 am (25908)

---

Yes, this is a necro-post, BUT: I found two games for the 3DS which have a csv attached to the archive with the TOC! I hope this helps to finally reverse the format!
Here's a sample pair: http://www80.zippyshare.com/v/UXIaJ2rk/file.html
Maybe somebody could first write a script that works with the csv and then try to do the same with the archive but without the CSV? That would be amazing!

aluigi, posted Sat Aug 26, 2017 10:28 am (25913)

---

Well done:
http://aluigi.org/bms/fuse_fib.bms

To be honest the format was not difficult so I think the main problem was the compression algorithm (rfpk) that has been implemented in quickbms only in the recent versions.

AlphaTwentyThree, posted Tue Aug 29, 2017 2:15 pm (26024)

---

Thanks a lot Luigi!
The version with the csv works like a charm.
However, there are some problems without csv:
http://www29.zippyshare.com/v/rM7jyW9i/file.html
http://www29.zippyshare.com/v/nWyvYhVR/file.html
First of all, the script terminates in an error.
Then, take look at file 33d - the file seems to contain several CWAV files which is strange in itself but then if you search for CWAV as a string you can see that the first three headers look correct and the fourth one is gibberish. Maybe there's something wrong with the SIZE/OFFSET pairs when extracting with no names? This could be what causes the crash at the end.

aluigi, posted Tue Aug 29, 2017 10:14 pm (26029)

---

There are two problems:

1) the files that cause the read error are those without the compression flags, in short in legoharrypotter_ipad.fib these files are NOT compressed and usually just a zero size while in project_silver_hh_3ds.fib they are compressed (even if they have no flag). Even in the same project_silver_hh_3ds.fib compressed and non compressed files are mixed! The result is that I have no way to know if they are compressed or not.

2) I don't know what's the compression algorithm in project_silver_hh_3ds.fib because no algorithm has been identified for the data after the 32bit compressed size field

The work-around I can adopt is just checking if the first 32bit of the non-compressed files (those without flags) is different than 0 and 1, in which case I will consider it a compressed file to dump "as-is" in a folder called "unknown_compression".
Script 0.1.2

Long story short: the script is 100% useless due to the files compressed with unknown compression, those encrypted with unknown encryption and moreover because the non-compressed files use their own format (with f1 ff ff ff or f2 ff ff ff at offset 0x10). What a waste of time...

AlphaTwentyThree, posted Wed Aug 30, 2017 11:39 am (26053)

---

Oh man... this really sucks. Sorry for the expression.
The format is so widely spread that an extraction possibility would really have been nice. But alas, it's how it is. Maybe Ekey has the patience to experiment further...
Thanks for all your work nevertheless. :\

AlphaTwentyThree, posted Sun Sep 03, 2017 10:11 pm (26212)

---

Ok, this may sound odd, but can you modify the script to the point where you dump all data, no matter if it's compressed or not? I'm only after the music which I hope is not compressed.
Sorry to bother you again with this but could you modify the script so it extracts all the contents of this file for example:
http://www42.zippyshare.com/v/lgGlwKRh/file.html
http://www42.zippyshare.com/v/KaN8PQtp/file.html
At least we have a better basis for further investigation then.

aluigi, posted Sun Sep 03, 2017 10:25 pm (26213)

---

Script 0.1.3, open it and set DISABLE_COMPRESSION to 1

AlphaTwentyThree, posted Mon Sep 04, 2017 11:24 am (26232)

---

Thanks for this!
There's some error hidden inside the script - the uncompressed files are sometimes truncated. Take this archive:
http://www12.zippyshare.com/v/xEGagmjn/file.html
http://www12.zippyshare.com/v/DEibBgqw/file.html
First of all, the script finishes with an error. Then, there are three examples for truncated files, the ones I found after a quick check: 8e, bb and 40b. They contain cwav files, which have the full file size at offset 0xc. The last file is truncated in all of these examples.
BTW, yes, the music is included in the files with known compression so your script is perfect once it functions properly.
Thanks for taking another look!

aluigi, posted Mon Sep 04, 2017 9:52 pm (26270)

---

No, really, this format is pure crap with every archive which is not compatible with the others, complete lack of reliability and any sense.
The only alternative is removing the script, I have chosen to add an additional useless work-around but I will no longer waste time here.

AlphaTwentyThree, posted Wed Sep 06, 2017 1:18 pm (26322)

---

Understandable. :\

AlphaTwentyThree, posted Mon Sep 25, 2017 4:14 pm (26815)

---

Maybe I'll try to implement more variants or rather write individual scripts for different games. I know that the demand is there, so why not do some good?