This topic was archived here:

https://web.archive.org/web/20230000000000fw_/https://www.zenhax.com/viewtopic.php?t=3971

swinei, posted Thu Mar 16, 2017 6:42 pm (21538)

---

I got the MEA EA and want to extract the files from it. This is the first time messing around with Frostbite 3 for me and I'm having trouble.

During my research I found the following helpful resources:

• http://aluigi.altervista.org/bms/frostbite.bms
• https://github.com/mitsuhiko/frostbite2-stuff
• https://github.com/GreyDynamics/Frostbite3_Editor

Questions:

• .toc files, wtf? What are they used for? The .sb files look more like a table of content. Are the .sb files enough to unpack?
• .toc encryption? In other games if the file is encrypted it starts with x00D1CE. In MAE it doesn't but always starts with x??D1 and ends with
  
  Code:

  B9 73 E9 D4 88 43 59 08 A5 9F 30 6A E4 D2 79 94
  B2 BB D1 7F 75 F4 D0 D1 01 00 00 40 65 21 61 64
  6E 58 64 24 5E 21 72 66 4F 73 72 44 79 49 72 49
  21 78 56 67 48 65 41 21 36 56 63

  are they encrypted? The D1 makes me think so.
• How does variable4 work? It says in the manual it's a Frostbite specific type.

I've uploaded a sample of .toc/.sb pairs here and a sample of .cas/.cat files here.

Any help is much appreciated

aluigi, posted Fri Mar 17, 2017 3:43 pm (21564)

---

These toc files look obfuscated yes.
Two weird things: the second byte is ever 0xd1 even if the others are ever different and that 0xd0 0xd1 0x01 0x00 0x00 "@e!adnXd$^!rfOsrDyIrI!xVgHeA!6Vc" at the end.

To know how the data is read in variable4 take a look at src\file.c searching "variable4":

Code:

            i = 0;
            do {
                c = fgetxx(fdnum, 1, error);
                if(*error) break;
                retn |= ((c & 0x7f) << i);
                i  = 7;
            } while(c & 0x80);

apollon08170, posted Fri Mar 17, 2017 9:44 pm (21583)

---

Hey i found this but don't work for me with extraction files =), if help you =)

http://files.1337upload.net/BF4_Python_ ... 18017e.rar

apollon08170, posted Fri Mar 17, 2017 10:16 pm (21584)

---

When i execute bf4dumper.py with files in path

Code:

#This script runs through all toc files it can find and uses that information to extract the files to a target directory.
#Often the assets are actually stored in cascat archives (the sbtoc knows where to search in the cascat), which is taken care of too.
#The script does not overwrite existing files (mainly because 10 sbtocs pointing at the same asset in the cascat would make the extraction time unbearable).

#Adjust paths here.

bf4Directory    = r"C:\Users\Apollon\Desktop\EXTRACTION\DATA"
targetDirectory = r"C:\Users\Apollon\Desktop\EXTRACTION\extract" #do yourself a favor and don't dump into the Users folder (or it might complain about permission) 


###The following paths do not require adjustments (unless the devs decided to rename their folders). Note that they are relative to bf4Directory.

#As files are not overwritten, the patched files need to be extracted first.
#The script will dump all tocs it can find in these two folders subfolders:
tocRoot  = r"Update" #patched and xpack files FIRST
tocRoot2 = r"Data"   #unpatched vanilla files SECOND

#Note: The "Update" tocRoot contains both patch (for vanilla AND xpack) and unpatched xpack files. The reason it still
#      works correctly is because it goes through the folders alphabetically, so the patch comes first.


#Feel free to comment out one or both cats if they don't exist (some Frostbite 2 games shipped without cats).
#Although in that case you could just as well use an invalid path, i.e. not change anything.
catPath       = r"\cas.cat" 
updateCatPath = r"\cas.cat"

.........
.....
...

The cas.cat file is "not found" ?

Code:

Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:42:59) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> 
========= RESTART: C:\Users\Apollon\Desktop\EXTRACTION\bf4dumper.py =========
Unpatched cat not found.
Patched cat not found.
>>> 

Do you have an idea ?

xrb936, posted Sun Mar 19, 2017 9:08 am (21618)

---

I tried to use the script for FIFA17, no good. Hope someone can find why!

AnonBaiter, posted Sun Mar 19, 2017 3:43 pm (21622)

---

I have never expected Bioware to get their hands on Frostbite 3 AGAIN.

If you want to dump just the .cas/.cat files then use my modified script. I want to know if it works on this "game" after you`ve done using it.

frostbite.rar

Snowpiercer, posted Sun Mar 19, 2017 6:59 pm (21631)

---

AnonBaiter wrote:

I have never expected Bioware to get their hands on Frostbite 3 AGAIN.

If you want to dump just the .cas/.cat files then use my modified script. I want to know if it works on this "game" after you`ve done using it.

Doesn't work for me, tries to open "cas_-446005142.cas" after pointing the script at a cas.cat file when the files are named cas_01.cas, cas_02.cas etc.

Code:

- enter in folder C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage
  coverage file 0     0%   0          8637248    . offset 00000000
- open input file C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage\cas.cat
- enter in folder C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage
- open input file C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage\cas_-446005142.cas

- error in src\file.c line 247: fdnum_open()
Error: No such file or directory

Last script line before the error or that produced the error:
  429 open FDSE TMP 1

Press ENTER or close the window to quit

AnonBaiter, posted Sun Mar 19, 2017 7:24 pm (21633)

---

Oh god.

Try to run quickbms through command-line and use -0 -V with the script, then the file in which you want to extract and post the results here after you`re done. After all I need to fix this even if I refuse to get the game by myself.

Snowpiercer, posted Sun Mar 19, 2017 8:00 pm (21634)

---

AnonBaiter wrote:

Oh god.

Try to run quickbms through command-line and use -0 -V with the script, then the file in which you want to extract and post the results here after you`re done. After all I need to fix this even if I refuse to get the game by myself.

Code:

C:\Users\Snowpiercer\Documents\QuickBMS>quickbms -0 -V "C:\Users\Snowpiercer\Documents\QuickBMS\Scripts\frostbite_mea.bms" "C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage\cas.cat" "C:\Users\Snowpiercer\Documents\MEA_Extract"

QuickBMS generic files extractor and reimporter 0.8.0
by Luigi Auriemma
e-mail: [email protected]
web:    aluigi.org
        (Mar 12 2017 - 15:03:46)

                   quickbms.aluigi.org  Homepage
                            zenhax.com  ZenHAX Forum
                               @zenhax  Twitter & Scripts

- current_folder: C:\Users\Snowpiercer\Documents\QuickBMS
- bms_folder:     C:\Users\Snowpiercer\Documents\QuickBMS\Scripts
- exe_folder:     C:\Users\Snowpiercer\Documents\QuickBMS
- file_folder:    C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage
- output_folder:  C:\Users\Snowpiercer\Documents\MEA_Extract
- temp_folder:    C:\Users\Snowpiercer\AppData\Local\Temp\
- open input file C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage\cas.cat
- open script C:\Users\Snowpiercer\Documents\QuickBMS\Scripts\frostbite_mea.bms
- set output folder C:\Users\Snowpiercer\Documents\MEA_Extract

  offset   filesize   filename
--------------------------------------
. 00000000 get     EXT        "cat" -1002
.
- enter in folder C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage
  coverage file 0     0%   0          8637248    . offset 00000000
- open input file C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage\cas.cat
. 0000022c idstr              "NyanNyanNyanNyan" 16
    4e 79 61 6e 4e 79 61 6e 4e 79 61 6e 4e 79 61 6e   NyanNyanNyanNyan
. 0000023c get     FILES      0x0003a923 4
. 00000240 getdstr ZERO       "" 20
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
    00 00 00 00                                       ....
. 00000244 getdstr HASH       "" 20
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
    00 64 1f 50                                       .d.P
. 00000258 get     FOFFSET    0xd288a5cd 4
. 0000025c get     FSIZE      0xf45b90c6 4
. 00000260 get     ZERO       0x2753e5e2 4
. 00000264 get     PACK_NUM   0xe56a806a 4
. 00000268 get     PACK_NAME  "cas" -1001
- enter in folder C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage
- open input file C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage\cas_-446005142.cas

- error in src\file.c line 247: fdnum_open()
Error: No such file or directory

Last script line before the error or that produced the error:
  429 open FDSE TMP 1

xrb936, posted Sun Mar 19, 2017 8:08 pm (21635)

---

AnonBaiter wrote:

Oh god.

Try to run quickbms through command-line and use -0 -V with the script, then the file in which you want to extract and post the results here after you`re done. After all I need to fix this even if I refuse to get the game by myself.

Code:

- open input file I:\Origin\Mass Effect Andromeda\Data\Win32\loc\en.toc
- open script C:\Users\Robin-PC\Desktop\quickbms\frostbite.bms
- set output folder O:\New folder

  offset   filesize   filename
--------------------------------------
- enter in folder I:\Origin\Mass Effect Andromeda\Data\Win32\loc
  coverage file 0     0%   0          115857     . offset 00000000
- open input file I:\Origin\Mass Effect Andromeda\Data\Win32\loc\en.TOC

- signature of 3 bytes at offset 0x00000000 doesn't match the one
  expected by the script:

  this one: "???"
  fd d1 e5                                          ...

  expected: ""
  00 d1 ce                                          ...

- 0 files found in 0 seconds
  coverage file 0     0%   3          115857     . offset 00000003

Press ENTER or close the window to quit

Here is what I got.

AnonBaiter, posted Sun Mar 19, 2017 9:11 pm (21636)

---

I`m working on it. Don`t count on me for .sb/.toc files though.

EDIT: Okay, on the frostbite.bms script, replace this line:

Code:

   if ZERO != "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
      goto -16 0 SEEK_CUR
   endif

With this line:

Code:

   if ZERO != "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
      goto -16 0 SEEK_CUR
   else
      goto 0 0 SEEK_CUR
   endif

See if it works.

Snowpiercer, posted Sun Mar 19, 2017 9:36 pm (21637)

---

AnonBaiter wrote:

I`m working on it. Don`t count on me for .sb/.toc files though.

EDIT: Okay, on the frostbite.bms script, replace this line:

Code:

   if ZERO != "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
      goto -16 0 SEEK_CUR
   endif

With this line:

Code:

   if ZERO != "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
      goto -16 0 SEEK_CUR
   else
      # do nothing
   endif

See if it works.

Getting the exact same output as before.

swinei, posted Sun Mar 19, 2017 9:54 pm (21638)

---

aluigi wrote:

These toc files look obfuscated yes.
Two weird things: the second byte is ever 0xd1 even if the others are ever different and that 0xd0 0xd1 0x01 0x00 0x00 "@e!adnXd$^!rfOsrDyIrI!xVgHeA!6Vc" at the end.

To know how the data is read in variable4 take a look at src\file.c searching "variable4":

Code:

            i = 0;
            do {
                c = fgetxx(fdnum, 1, error);
                if(*error) break;
                retn |= ((c & 0x7f) << i);
                i  = 7;
            } while(c & 0x80);

Thanks, how can I deobfuscate the .toc files? And can you tell me what is in them that isn't in the .sb files?

apollon08170 wrote:

Hey i found this but don't work for me with extraction files =), if help you =)

http://files.1337upload.net/BF4_Python_ ... 18017e.rar

I checked that out thank you.

AnonBaiter wrote:

I have never expected Bioware to get their hands on Frostbite 3 AGAIN.

If you want to dump just the .cas/.cat files then use my modified script. I want to know if it works on this "game" after you`ve done using it.

Can you tell me how to save the bundles? What's up with ebx, dbx, res, chunk, bmm?
And can you tell me what's usually in the .toc files? And why is there so many duplicates?

I can read cas.cat files and .sb files with the attached project, it's a VS C# program. Currently working on writing the bundles to disk.

MEA Unpack.zip

AnonBaiter, posted Sun Mar 19, 2017 10:12 pm (21639)

---

swinei wrote:

Can you tell me how to save the bundles? What's up with ebx, dbx, res, chunk, bmm?
And can you tell me what's usually in the .toc files? And why is there so many duplicates?

What? I don't have the game, and I won't intend to have it anytime soon. You don't want to know why.

swinei, posted Sun Mar 19, 2017 10:22 pm (21640)

---

AnonBaiter wrote:

What? I don't have the game, and I won't intend to have it anytime soon. You don't want to know why.

Now I'm really curios why not?

You seem to know the Frostbite engine that's why I ask. I don't have any other Frostbite games to look at. I can also upload more sample files if you need anything specific.

AnonBaiter, posted Sun Mar 19, 2017 11:06 pm (21642)

---

swinei wrote:

You seem to know the Frostbite engine that's why I ask. I don't have any other Frostbite games to look at.

Well, to be honest though I'm just a beginner on this stuff.

swinei wrote:

I can also upload more sample files if you need anything specific.

Thanks for the offer, but I already have a Frostbite 3 game on my HDD. Which is enough since my Frostbite "research" is limited to games made by EA DICE with that engine. That habit of mine might change sometime though, just not today.

xrb936, posted Mon Mar 20, 2017 3:41 am (21643)

---

AnonBaiter wrote:

swinei wrote:

You seem to know the Frostbite engine that's why I ask. I don't have any other Frostbite games to look at.

Well, to be honest though I'm just a beginner on this stuff.

swinei wrote:

I can also upload more sample files if you need anything specific.

Thanks for the offer, but I already have a Frostbite 3 game on my HDD. Which is enough since my Frostbite "research" is limited to games made by EA DICE with that engine. That habit of mine might change sometime though, just not today.

So is there any possible to extract files from toc and sb files?

AnonBaiter, posted Mon Mar 20, 2017 3:54 am (21644)

---

No ideas at the moment.

aluigi, posted Mon Mar 20, 2017 7:45 am (21646)

---

AnonBaiter wrote:

if ZERO != "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"[/code]See if it works.

That can't work because you can't use C escapes in conditions.
Strings containing escapes are allowed only in those few commands where it's written "cstring" in quickbms.txt (unfortunately this is an "heritage" of the original bms language).

If TOC is encrypted there is no way to extract data from it using the current script.

swinei, posted Tue Mar 21, 2017 8:28 am (21679)

---

The CAS entries are compressed with Zstandard. Yay comptype scanner
compression type = 0F 70

The resId for textures is 1809719482

So now I have the res entry with the DDS header info and the chunk entry with the texture data.

The DDS header info has the 16 byte chunk id at position 32.
The last 16 bytes is maybe the name, but multiple entries can have the same name hm.
Width and height shorts start at 22 (I think).

I've attached an example pair, any help figuring out the header would be much appreciated!

edit: I started on a 010 template with info from here and here

Code:

//------------------------------------------------
//--- 010 Editor v7.0.2 Binary Template
//
//      File: 
//   Authors: 
//   Version: 
//   Purpose: 
//  Category: 
// File Mask: 
//  ID Bytes: 
//   History: 
//------------------------------------------------

int mipOneEndOffset;
int mipTwoEndOffset;
int textureType;
int pixelFormat;
short unknown;

short depth;
short sliceCount;

short width;
short height;

byte numSizes;
byte firstMip;

int dummy;

char chunkID[16];

local int i;
for(i=0;i    int mipSize;
}

int chunkSize;

byte dummyy[16];

int nameHash;
char name[16];

AnonBaiter wrote:

Well, to be honest though I'm just a beginner on this stuff.

Ha me too

AnonBaiter wrote:

Thanks for the offer, but I already have a Frostbite 3 game on my HDD. Which is enough since my Frostbite "research" is limited to games made by EA DICE with that engine. That habit of mine might change sometime though, just not today.

Could you upload some decrypted .toc files please? I wanna know whats in them

AnonBaiter, posted Tue Mar 21, 2017 9:20 am (21682)

---

swinei wrote:

Could you upload some decrypted .toc files please? I wanna know whats in them

From this game? No, I don`t have them.

volfin, posted Tue Mar 21, 2017 4:36 pm (21688)

---

The data of your sample is straight DTX5.



However, it seems to only be MIPS. Your header says it should be 1024x1024, but it only contains 256x256 and below. So there must be another data chunk of the highest levels of the texture. Or something in the header to indicate those levels are absent. Some engines allow for MIP level discarding, it's how they tweak memory usage before release.

You would need to look across a large # of textures to see which bytes change in the headers to figure out which signal what the format is, one texture isn't enough to do that.

pyr0ball, posted Wed Mar 22, 2017 7:12 pm (21705)

---

Hello gentlemen.

I came across this thread just googling for a way to extract assets from MEA. I'm particularly after high quality meshes for use in 3D modelling.

I'm a quick study on hacks and code, but pretty much new to extracting/hacking assets from video games. I see there's some tools already in play here, but it looks like there's some backend knowledge on usage I haven't looked at yet. I'll search around on these forums for tutorials.

In the meantime, has anyone located which archive the weapon models are in, and possibly could provide a best practice for extracting meshes?

angelus2402004, posted Sat Mar 25, 2017 2:13 pm (21753)

---

Has anyone found a way to extract the music files?

thanks