This topic was archived here:

https://web.archive.org/web/20230000000000fw_/https://www.zenhax.com/viewtopic.php?t=8678

CosmicDreams, posted Fri Oct 05, 2018 1:49 pm (39116)

---

Okay so first things first, yes i tried the bms script made for the Division and yes it mostly works but there are issues.

Basically, it eventually runs into an error like this:

Code:

- open input file C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data\sdf-C-2265.sdfdata
  00000000 119        moria/baked/art/[assets]/[generictexture]/gn_roof_tile_bitumen_01_d.dds

- The following output file already exists:
  moria\baked\art\[assets]\[generictexture]\gn_roof_tile_bitumen_01_d.dds
  Do you want to overwrite it?
    y = overwrite (you can use also the 'o' key)
    n = skip (default, just press ENTER)
    a = overwrite all the files without asking
    r = automatically rename the files with the same name
    s = skip all the existent files without asking

Some files are split across the SDFDATA archives and the script runs into issues with this, but
no matter what option you choose here it immediately goes into this:

Code:

Error: incomplete input file 1: C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data\sdf-C-2265.sdfdata
       Can't read 114 bytes from offset 00000005.
       Anyway don't worry, it's possible that the BMS script has been written
       to exit in this way if it's reached the end of the archive so check it
       or contact its author or verify that all the files have been extracted.
       Please check the following coverage information to know if it's ok.

  coverage file 1   100%   5          5          . offset 00000005

Last script line before the error or that produced the error:
  127 log name packageOffset decompressedSize 1

The script also doesn't support the merging of DDS headers from the SDFTOC file to the output DDS files in the extracted SDFDATA, so it has to be done manually at the moment, Although theres a whole other issue with those that i'd need to go post about on the Graphics Files board instead(Basically its an obscure DXGI format that uses ASTC i think)

Running the script on smaller amounts of SDFDATA at a time seems to help with the issues a bit but the file splitting is still an issue.
One of my friends was trying to modify the tool the script is based off (Rogue_SDF) to work with this game but ran into issues with "chunks"? That's all i know.
I'll send samples if required but i think a full dump may be needed to properly investigate the issues.

Any help would be appreciated
In the meantime here is a screenshot of one of the models that managed to extract properly and load in Noesis.

aluigi, posted Fri Oct 05, 2018 4:35 pm (39125)

---

Do you mean you have already tried the http://aluigi.org/bms/ubisoft_sdf.bms script?

CosmicDreams, posted Fri Oct 05, 2018 4:50 pm (39128)

---

Yes, that's the one.

aluigi, posted Fri Oct 05, 2018 5:05 pm (39131)

---

Could you please upload the sdftoc file?
Basically sdftoc is telling you that sdf-C-2265.sdfdata contains a file with a duplicated name and a size of 114 bytes while that "container" is 5 bytes only, so there is something weird or something else to implement.

CosmicDreams, posted Fri Oct 05, 2018 5:36 pm (39133)

---

https://cdn.discordapp.com/attachments/419711036837330956/497824314033438723/sdf.sdftoc
It's zstd compressed if i remember correctly.

aluigi, posted Fri Oct 05, 2018 6:09 pm (39136)

---

Long story short, I don't know what's the problem.

Everything is correct till the code "for chunkIndex = 0 So what I did was ignoring the instructions if ch3 is 0x00 (which is an invalid value indeed) and apparently everything works correctly
Script 0.1.3a, let me know if you get other errors.

CosmicDreams, posted Fri Oct 05, 2018 6:19 pm (39139)

---

Same errors as before but it happens slightly later i think.

CosmicDreams, posted Fri Oct 12, 2018 2:19 pm (39455)

---

Okay, i forgot about another error that happens if it doesn't run into any existing files. .

Code:

Error: incomplete input file -1:
       Can't read 3 bytes from offset 00527172.
       Anyway don't worry, it's possible that the BMS script has been written
       to exit in this way if it's reached the end of the archive so check it
       or contact its author or verify that all the files have been extracted.
       Please check the following coverage information to know if it's ok.

  coverage file -1  117%   6355722    5402994    . offset 00527172

Last script line before the error or that produced the error:
  87  get fileId long MEMORY_FILE

aluigi, posted Fri Oct 12, 2018 5:00 pm (39464)

---

Yesterday I updated the script for supporting version 0x17 and it may fix that error.
Anyway don't worry since it just means you have extracted everything and you can ignore it.

CosmicDreams, posted Fri Oct 12, 2018 7:54 pm (39472)

---

The issue with files split across archives is still there

CosmicDreams, posted Sat Oct 13, 2018 8:35 am (39479)

---

The other issue is gone, forgot to mention that somehow.
.

aluigi, posted Sat Oct 13, 2018 10:05 am (39482)

---

Can you remind me what's the issue with "files split across archives"?
The script supports the various sdfdata and I'm not aware of files that are located half in one and half in another archive.

CosmicDreams, posted Sat Oct 13, 2018 11:38 am (39489)

---

Well one would be Mario's model, i think.

Code:

- enter in folder C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data
- open input file C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data\sdf-A-0208.sdfdata
  00000000 0          moria/baked/art/animation/01-chr/02-her/mario/chr_her_mario_tpose_01.mmb
  0001b283 65536      moria/baked/art/animation/01-chr/02-her/mario/chr_her_mario_tpose_01.mmb
  00021b5c 65536      moria/baked/art/animation/01-chr/02-her/mario/chr_her_mario_tpose_01.mmb
  000294fa 30365      moria/baked/art/animation/01-chr/02-her/mario/chr_her_mario_tpose_01.mmb
- enter in folder C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data
  coverage file 1     0%   72403      24936897   . offset 00000000
- open input file C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data\sdf-A-0209.sdfdata
  000092f9 2230304    moria/baked/art/animation/01-chr/02-her/mario/chr_her_mario_tpose_01.mmb

- The following output file already exists:
  moria\baked\art\animation\01-chr\02-her\mario\chr_her_mario_tpose_01.mmb
  Do you want to overwrite it?
    y = overwrite (you can use also the 'o' key)
    n = skip (default, just press ENTER)
    a = overwrite all the files without asking
    r = automatically rename the files with the same name
    s = skip all the existent files without asking

And no matter what you choose this happens:

Code:

Error: incomplete input file 1: C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data\sdf-D-3846.sdfdata
       Can't read 10341 bytes from offset 00000005.
       Anyway don't worry, it's possible that the BMS script has been written
       to exit in this way if it's reached the end of the archive so check it
       or contact its author or verify that all the files have been extracted.
       Please check the following coverage information to know if it's ok.

  coverage file 1   100%   5          5          . offset 00000005

Last script line before the error or that produced the error:
  155 clog name packageOffset compSize CHUNK_SIZE 1

Press ENTER or close the window to quit

CosmicDreams, posted Sat Oct 13, 2018 11:44 am (39490)

---

SDFDATA 3846 seems to be a 5 byte dummy file that apparently contains 2 parts of a model???
Something is really odd here.

Code:

- open input file C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data\sdf-D-3846.sdfdata
  00000000 0          moria/baked/art/animation/01-chr/03-enm/globetrotter/chr_enm_shd_globetrotter_tpose_01.mmb
  00000000 65536      moria/baked/art/animation/01-chr/03-enm/globetrotter/chr_enm_shd_globetrotter_tpose_01.mmb

CosmicDreams, posted Sat Oct 13, 2018 11:50 am (39491)

---

Removing all the dummy files still ends up with a similar issue,

Code:

- open input file C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data\sdf-B-1702.sdfdata
  00c79b12 54030      moria/baked/art/animation/01-chr/03-enm/shielder/chr_enm_lvp_shielder_tpose_01.mmb
- enter in folder C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data
  coverage file 1     0%   24139      25137233   . offset 00000000
- open input file C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data\sdf-B-1703.sdfdata
  000081da 12257012   moria/baked/art/animation/01-chr/03-enm/shielder/chr_enm_lvp_shielder_tpose_01.mmb

Error: incomplete input file 1: C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data\sdf-B-1703.sdfdata
       Can't read 101529 bytes from offset 008ef541.
       Anyway don't worry, it's possible that the BMS script has been written
       to exit in this way if it's reached the end of the archive so check it
       or contact its author or verify that all the files have been extracted.
       Please check the following coverage information to know if it's ok.

  coverage file 1    99%   9335655    9368897    . offset 008ef541

Last script line before the error or that produced the error:
  140 log name packageOffset decompressedSize 1

Press ENTER or close the window to quit

aluigi, posted Sat Oct 13, 2018 2:16 pm (39498)

---

I already provided a full explanation of why it's impossible to extract N bytes from a dummy archive containing only 5 bytes.

aluigi, posted Sat Oct 13, 2018 2:43 pm (39500)

---

Let's say that ch3 equal than zero should be handled as a "break" instead of a "continue" (like it's right now).
I have updated the script for trying this other method and fixing an unrelated bug in DUMP_FILE.
Let me know if the situation is now better.

CosmicDreams, posted Sat Oct 13, 2018 3:42 pm (39503)

---

Still getting the

Quote:

CHUNK_SIZE 1

error from it trying to extract from a dummy file, so no change from the previous version basically.

Very confusing issue...
I'll upload a batch of samples shortly. (i already have the entire game uploaded to google drive so if the samples aren't enough help i can pm a link to that or something, don't wanna link something like that publicly)

CosmicDreams, posted Sat Oct 13, 2018 3:52 pm (39505)

---

Alright here's the samples.
There was a file in a folder called chunkmapping.mchunks at the

Quote:

romfs\moria\sdfconfig\nx

directory that i thought i should include.
Everything else is from the

Quote:

romfs\moria\sdf\nx\data

directory, the sdfver file is probably useless but i included it anyway.
https://drive.google.com/file/d/1HSKLQl ... sp=sharing

EDIT: Of course i forgot the link.

CosmicDreams, posted Sat Oct 13, 2018 4:09 pm (39506)

---

https://cdn.discordapp.com/attachments/419711036837330956/500701420807258113/sdf-slice.mapping
This might be helpful too.

CosmicDreams, posted Sun Oct 14, 2018 12:40 am (39524)

---

Was chatting with someone else earlier about this.
https://cdn.discordapp.com/attachments/419711036837330956/500829671122927617/Screenshot_20181014-013602_Discord.jpg

Maybe this information will be of help?

aluigi, posted Sun Oct 14, 2018 2:25 pm (39548)

---

Currently I can't work on that, sorry.
If you find a solution feel free to post the full details or the bms patch.

Qnoops, posted Sun Oct 14, 2018 5:09 pm (39563)

---

aluigi wrote:

Long story short, I don't know what's the problem.

Everything is correct till the code "for chunkIndex = 0 So what I did was ignoring the instructions if ch3 is 0x00 (which is an invalid value indeed) and apparently everything works correctly
Script 0.1.3a, let me know if you get other errors.

So you updated the current ubisoft_sdf script that was working with south park game, now will it still work with the south park game? did you added new specs or completely changed it? : D

Just in case i backed up old script working with SP game

CosmicDreams, posted Sun Oct 14, 2018 5:33 pm (39566)

---

Should still work i guess.
Doesn't seem like the script was changed just added to.