This topic was archived here:

https://web.archive.org/web/20230000000000fw_/https://www.zenhax.com/viewtopic.php?t=11

aluigi, posted Tue Aug 05, 2014 1:44 pm (14)

---

Ubisoft use a specific file format for their games, you can recognize it by the 2TAF and 3TAF magic signature at the beginning of the archive.

The format is known to be used by:
FarCry 3 - FAT2
The Crew - FAT2
Watch Dogs - FAT3

The format is divided in two files:
- FAT containing the index table
- DAT containing only the data referenced by FAT

One of the bad things of formats used by multiple games and multiple platforms is the existence of multiple customizations.
For example the same archive on Xbox360, PS3 and PC is handled in different ways.

This difference is related mainly to the handling of compressed files because FAT3 on Xbox uses LZX (aka xmem), PC too (it's rare, maybe they use the same DX sdk) and PS uses zlib.
To make the situation even more difficult is the fact that the compression is chunked so there is no compressed and uncompressed size but a sequence of compressed parts of the file.

FAT2 is easier because doesn't use chunks but also in that case there are at least 2 different compression algorithms: lzo and lzma.

A bad, very bad, thing of this format is that its files are nameless, there is only a name CRC to identify them.

There are various tools to handle the format for specific games, moreover due to the name CRC limitation.

I worked on FAT2/3 just recently and I release a script for quickbms that should be compatible with all the games and all or almost all the platforms:
http://aluigi.org/papers/bms/fat2_fat3.bms

For reference, the following is a FAT2 file from The Crew:

Code:

32 54 41 46 05 00 00 00 01 00 00 00 2a 10 00 00   2TAF........*...
61 0d 00 00 00 00 00 00 10 d1 b9 82 73 fa 06 00   a...........s...
16 02 00 c0 3c 70 62 00 b1 e3 01 00 00 00 00 00   ....30 6c d8 3c 14 af 15 00 73 4e 00 40 a7 a5 90 00   0l.<....sN.@....
61 3a 00 00 00 00 00 00 b5 dc b4 62 73 27 1d 00   a:.........bs'..
1d 08 00 c0 d7 a3 24 00 e1 b4 00 00 00 00 00 00   ......$.........
cb b5 da 79 88 ec 2d 00 cc 16 00 c0 f4 a7 57 00   ...y..-.......W.
91 f1 00 00 00 00 00 00 c2 a6 dc 01 56 e1 37 00   ............V.7.
8b 21 00 40 24 f6 79 00 81 ad 0a 00 00 00 00 00   .!.@$.y.........
76 a5 e1 df a6 dd 48 00 f9 64 02 40 c4 be 0b 00   v.....H..d.@....

While the following is a FAT3 from Watch Dogs:

Code:

33 54 41 46 08 00 00 00 03 04 37 00 ab 12 00 00   3TAF......7.....
3b 06 23 00 82 05 02 00 00 11 00 80 1a 9d 97 00   ;.#.............
5c 02 30 00 82 05 02 00 f0 11 00 80 c2 54 97 00   \.0..........T..
3f 99 3e 00 82 85 00 00 30 03 00 e0 4b 24 17 00   ?.>.....0...K$..
9b 7d 52 00 00 00 00 00 a0 01 00 00 7d 03 20 00   .}R.........}. .
28 0e 63 00 82 05 02 00 30 11 00 80 20 0c 97 00   (.c.....0... ...
fa f3 67 00 82 05 08 00 c0 46 00 80 2d 23 83 00   ..g......F..-#..
95 04 6a 00 82 05 10 00 a0 89 00 80 33 5d 82 00   ..j.........3]..
cf 5f 6a 00 82 05 10 00 a0 cf 00 80 23 fa 8b 00   ._j.........#...
7b a3 6e 00 0a 1a 04 00 11 34 00 60 4c 0a 1f 00   {.n......4.`L...

cenkzenk, posted Fri Aug 22, 2014 10:34 am (229)

---

Hi How to open .loc file

CobraGamer, posted Sun Dec 09, 2018 7:58 pm (41162)

---

Could you take a look at the format on The Crew 2 archives? This BMS script doesn't work with them.
I'd be really thankful if you could release an updated script for this!

aluigi, posted Mon Dec 10, 2018 11:12 am (41209)

---

oodle compression, script 0.1.5:
http://aluigi.org/bms/fat2_fat3.bms

CobraGamer, posted Mon Dec 10, 2018 12:16 pm (41215)

---

Thank you!

cyberspeed, posted Sat Nov 13, 2021 7:37 pm (67492)

---

aluigi wrote:

... A bad, very bad, thing of this format is that its files are nameless, there is only a name CRC to identify them.
...

Would it be a very bad idea if the script could extract them for their names as they are now but with their CRC64Bit hashes?
Would help a lot to identify them would be very helpful if you could update the script to support that.

aluigi, posted Sat Nov 13, 2021 8:22 pm (67494)

---

Sure, at line 244 replace 'set NAME string ""' with:

Code:

xmath TMP_CRC "NAME_CRC >> 32"
string NAME p "xx" TMP_CRC NAME_CRC

Additionally I'm quite sure there is a bug at line 75 where "get DUMMY longlong" should be "get NAME_CRC longlong" other version 9 wouldn't have the correct crc associated.
Please let me know how it works.

cyberspeed, posted Sun Nov 14, 2021 3:46 pm (67507)

---

aluigi wrote:

Sure, at line 244 replace 'set NAME string ""' with:....
...a bug at line 75 where "get DUMMY longlong" should be "get NAME_CRC longlong" ...
Please let me know how it works.

I would like to confirm both recommendations above work as expected, and the results are accurate. Thank you so much. The script can definitely be updated for future use that way no one will get confused what the files are as i did
Before:

Code:

  Offset           Filesize   Name
  00000000170f8ea7 263333     0000000000000000.dat
  000000003e7acfa3 22         0000000000000001.dat
  00000000247ba630 485613     0000000000000002.dat
  0000000083451d28 134246     0000000000000003.dat
  00000000836031e7 263333     0000000000000004.dat

After:

Code:

  Offest           FileSize   CRC64
  00000000170f8ea7 263333     0002CE399C7095C2
  000000003e7acfa3 22         00035ACDB463C178
  00000000247ba630 485613     0005161C58F1FAE8
  0000000083451d28 134246     000A94756572D236
  00000000836031e7 263333     000E9033917CE2FF

All these Ubisoft games built on Dunia engine, have actual file names but need to be manually edited/added, so how can i feed the list with the names so the assets can be extracted with file/folder structure based on their reversed hashes?

Example hashes above would be extracted like so in their file/folder structure:

Code:

project\road66\base\area\area_01\area_01_06\secteur_01_06_06_00\detailid_01_06_06_00.xbt
project\road66\generated\area\area_02\area_02_10\secteur_02_10_05_01\ai_02_10_05_01.van
project\road66\generated\area\area_01\area_01_09\secteur_01_09_07_01\ai_01_09_07_01.van
project\road66\generated\area\area_05\area_05_11\secteur_05_11_02_03\displacement_05_11_02_03.hms
project\road66\base\area\area_05\area_05_11\secteur_05_11_02_05\detailid_05_11_02_05.xbt

I tried something like:

Code:

quickbms.exe script.bms filenames.txt input output

And few more variations, but to no avail, is there a way to do it? I cant figure it out

aluigi, posted Sun Nov 14, 2021 7:17 pm (67513)

---

Perfect.

For that feature you mention exists the NameCRC command.
I don't know what crc/hash algorithm is used on these filenames and if it's supported by the crc engine of quickbms.
If you know the algorithm filenames.txt must contain only the filenames, otherwise it must contain a hexadecimal crc followed by the name (like "1122334455667788 path\file.txt").

In that case would be enough to replace the whole content of the GET_NAME_FROM_CRC function with:
namecrc NAME NAME_CRC "filenames.txt" 64

cyberspeed, posted Sun Nov 14, 2021 10:56 pm (67518)

---

aluigi wrote:

Perfect....I don't know what crc/hash algorithm is used on these filenames and if it's supported by the crc engine of quickbms....

Based on: http://aluigi.altervista.org/bms/quickb ... engine.txt looks like its supported, its the Jones variant

aluigi wrote:

In that case would be enough to replace the whole content of the GET_NAME_FROM_CRC function with:
namecrc NAME NAME_CRC "filenames.txt" 64

I did try, yet every single assets gets overwritten by a name like: (2).bin
And not sure i fully understand how to modify the function
Te names inside the filenames.txt are with out any other characters besides the actual path as described above for those 5 CRC64 hashes, no ""

aluigi, posted Mon Nov 15, 2021 5:16 pm (67527)

---

So I guess this is the correct command:
namecrc NAME NAME_CRC "filenames.txt" 64 0xad93d23594c935a9 "64 0 0 0 1 0"

Regarding the function I meant, from:

Code:

startfunction GET_NAME_FROM_CRC
    set NAME string ""
    if NFO_EXISTS != 0
        for z = 0 < NAMES
            getarray TMP_CRC 10 z
            if TMP_CRC == NAME_CRC
                getarray NAME 11 z
                math z = NAMES  # break is bugged
            endif
        next z
    endif
endfunction

To:

Code:

startfunction GET_NAME_FROM_CRC
    namecrc NAME NAME_CRC "filenames.txt" 64 0xad93d23594c935a9 "64 0 0 0 1 0"
    if NAME == ""
        xmath TMP_CRC "NAME_CRC >> 32"
        string NAME p "xx" TMP_CRC NAME_CRC
    endif
endfunction

cyberspeed, posted Tue Nov 16, 2021 9:45 pm (67549)

---

Thank you, i tested both TC games and it works flawlessly.

cyberspeed, posted Fri Mar 18, 2022 10:59 pm (70595)

---

The script will NOT export strings that happen to have a space in the entire full path, it gets skipped, how can i avoid that?
Ideal would be to take in to consideration entire string line, spaces or not.
Can that be achieved?

cyberspeed, posted Mon Apr 04, 2022 7:23 pm (70858)

---

Would anyone else happen to know how to make the script extract the strings that have spaces in name at all?
Right now the whole string is skipped.

spiritovod, posted Mon Apr 04, 2022 9:54 pm (70859)

---

I suppose more explanation about the issue is required. I think cyberspeed is talking about situation, when list with filenames contains strings with spaces, but CRC is not calculated properly for them. In the same time, explicit declaration works fine. For example, for CRC32 with default parameters:
[extracted\test file.bin] with NAME_CRC = 0x402C6A32 -> doesn't work (output will be 402C6A32.dat)
[0x402C6A32 extracted\test file.bin] with NAME_CRC = 0x402C6A32 -> works fine (output will be "extracted\test file.bin")

@cyberspeed: If you can calculate those crc manually, you can use list with pre-defined crc values, this way it will work as expected.

cyberspeed, posted Tue Apr 05, 2022 8:16 pm (70880)

---

spiritovod wrote:

...I think cyberspeed is talking about situation, when list with filenames contains strings with spaces, but CRC is not calculated properly for them...

That is correct.

spiritovod wrote:

@cyberspeed: If you can calculate those crc manually, you can use list with pre-defined crc values, this way it will work as expected.

I always assumed the script is written to not support spaces, or that it would require more code to support them, using your suggestion works as it should, i just had no idea it would require the HASH to be included even though quickbms natively supports that CRC64 algo. Thanks non the less.
I was confused when Luigi said above, if the CRC is unknown the CRC64 hash must be provided but if its supported by quickbms it is not necessary, so now i know.

spiritovod, posted Tue Apr 05, 2022 9:14 pm (70882)

---

@cyberspeed: Um, no, it's more like a quickbms bug. Like explained in the documentation, you may use list with paths only and list with pre-defined values paths, both should work in the same way for supported crc schemes. For example:
[extracted\test2.bin] with NAME_CRC= 0x2AB02897 -> works fine (output will be "extracted\test2.bin")

I didn't dig into quickbms code, but it may happen if you'll use space as delimiter for detecting pre-defined value in a string, or something. For example, you have [0x402C6A32 extracted\test file.bin] and you need to know, if it contains pre-defined value or not - in that case you search for the first space and split the string into pre-defined value and path. But what if you have only path with a space - such string will be splited in the same way (though it shouldn't). It's just a guess though.

Davida, posted Thu Jun 23, 2022 9:22 am (72252)

---

Could you take a look at the format on X360 Far Cry 4 archives? This BMS script doesn't seem to work with them.
I'd be really thankful if you could release an updated script for this!

Sample file, 6MB: https://ufile.io/t5w8iy65

h3x3r, posted Thu Jun 23, 2022 10:00 am (72253)

---

@Davida where did you get that *.NFO file? Beacause it contains filenames hashed version. Post all of them if there are any others.

FaisedWorld, posted Fri Jul 15, 2022 1:52 pm (72612)

---

Hey, can somebody help me with reimporting The Crew 1 .dat files? I figured out the basics of exporting and reimporting but something doesn't seem right when reimporting a single file in a .dat file. I tried different script versions (0.1.5 / 0.1.6) and they give different results, none of them working.

0.1.5 seems to work but at the end of the reimported file there seems to be a spam of "DC1|NUL|NUL|NUL|NUL|NUL|NUL..." and so on, game doesn't open and crashes, while on 0.1.6 the entire reimported file is "DC1|NUL|NUL|NUL|NUL...", I'm using notepad to view btw.

The file I'm trying to mess with is from startup.dat from the original 2014 The Crew files. To be specific I'm trying to change a single 165KB xml file called "00000042.xml" with 0.1.5 or "683eecf8683eecf8" with 0.1.6, which is the network / server config file. It seems to try to connect to an older IP compared to the up-to-date TC1 startup.dat and I'm trying to make the original 2014 crew connect to the current TC1 servers in hopes that maybe I can get past the title screen and the #0_1 error it gives me when trying to enter the game. So basically I'm just trying to see if I can somehow play the original crew, which I know is unlikely but still, the least I can do is try because I really want to.

(If anyone's curious about the specific IPs: the up-to-date crew connects to 185.38.20.70:4000 when pressing any key on title screen, while the 2014 crew connects to 194.169.249.154:4000 which is outdated)


If anyone can help me with this reimporting problem then please do
Thanks!