Helldivers 2 Model Extraction Help!

[Moo]

Edit NVM, I'm dumb, answered my own question

Edited March 6 by Moo

[HW12Dev]

On 3/4/2024 at 1:21 PM, SiNDoL said:

someone is working on stingray explorer here : https://github.com/HW12Dev/Stingray-Explorer

Hey, that's me!

If anyone has any questions about anything my tool does/cant do yet. Or any question about Stingray's internals, feel free to reach out and I can try my best to answer!

[tugoose]

3 hours ago, HW12Dev said:

Hey, that's me!

If anyone has any questions about anything my tool does/cant do yet. Or any question about Stingray's internals, feel free to reach out and I can try my best to answer!

 

Hi, I just stumbled on this forum whilst looking for Helldivers 3D Models. I'm a filmmaker and not really too well versed in the whole scripting or coding side of things. I plan to use a bunch of 3d models and possibly sound effects extracted from the game to make a short film.

I assume your tool will help me do that in an easy way or am I misunderstanding how it works? Ideally I'd be able to extract either ALL model files or certain categories of files from the game and save them to a local drive. After which I could sort and delete those I don't need.

Any insight or even currently applicable advice(dumbed down for a non-coder) would be appreciated

[CosmicDreams]

Hey this seems to be the only HD2 thread on here so i'll ask here

Does anyone know how wwise works with this engine?

The wwise_stream files should have wwise ids matched to them somewhere somehow,

for instance i think... acbe70e124e435d4.wwise_stream = 956335136.wem

i've tried a lot of different things but no dice so far...

[FunnyML]

55 minutes ago, CosmicDreams said:

The wwise_stream files should have wwise ids matched to them somewhere somehow,

for instance i think... acbe70e124e435d4.wwise_stream = 956335136.wem

Maybe they are referenced directly in the BNK files? I didn't manage to decrypt those banks with Wwiser to verify even though @nblock provided a key, maybe it works on your end?

Edited March 9 by FunnyML

[CosmicDreams]

11 minutes ago, FunnyML said:

Maybe they are referenced directly in the BNK files? I didn't manage to decrypt those banks with Wwiser to verify even though @nblock provided a key, maybe it works on your end?

I think i can get all the actual stingray hash names for wems anyway, the path is just content/audio/[wemid]

gonna see if i can automate it and add em all to the hash lists

[HW12Dev]

12 hours ago, tugoose said:

 

Hi, I just stumbled on this forum whilst looking for Helldivers 3D Models. I'm a filmmaker and not really too well versed in the whole scripting or coding side of things. I plan to use a bunch of 3d models and possibly sound effects extracted from the game to make a short film.

I assume your tool will help me do that in an easy way or am I misunderstanding how it works? Ideally I'd be able to extract either ALL model files or certain categories of files from the game and save them to a local drive. After which I could sort and delete those I don't need.

Any insight or even currently applicable advice(dumbed down for a non-coder) would be appreciated

At the moment, the tool just extracts all the files it can. In the future, there will be a UI for which files you want to extract, and hopefully by then we'll have the non-hashed file names for models, so you can extract just the ones you want.

[tugoose]

9 hours ago, HW12Dev said:

At the moment, the tool just extracts all the files it can. In the future, there will be a UI for which files you want to extract, and hopefully by then we'll have the non-hashed file names for models, so you can extract just the ones you want.

Awesome, so "all files it can" would likely include some 3d files and textures with correct UVs? possibly some sound effects? If so I very much look forward to it and would love to credit you in the description should your tool help me pull this off.

Best of luck!

[tugoose]

Okay so with much effort and help from a friend that is more capable, I've managed to extract all 51k game files. I still have no idea how to convert anything into usable formats such as .obj or images and textures that can be viewed. AFAIK this is not possible yet? or am I missing somehting?

On 2/29/2024 at 6:33 AM, indy_ said:

Question on using the new Extractor: I'm hunting for .obj files, and I've been using your .bat method from a few days ago to accomplish this piecemeal, section by section. However, I'm trying to do the same act in one fell swoop across all files in 'data'. This is where I'm at right now:

hellextractor extract -f ".*\.obj$" -o "output" "source"

Running this doesn't output .obj's from the test files I'm running it on. I can still find the .obj's using the older method (works fine, I've actually 3d printed a few of my finds already), but I'm hoping to use the updated Hellextractor to do it all in one fell swoop. Any advice?

Hey man, I tried the only method I can figure out from this forum post. When you say "older method" that you used to get the OBJ files, what method are you specifically referring to and could you give me a rough walk through of how to accomplish this.

[CosmicDreams]

23 hours ago, CosmicDreams said:

I think i can get all the actual stingray hash names for wems anyway, the path is just content/audio/[wemid]

gonna see if i can automate it and add em all to the hash lists

Managed to do it! made a PR for Hellextractor if anyone wants to grab the names for their own hash lists ❤️

[indy_]

On 3/9/2024 at 12:38 PM, tugoose said:

Okay so with much effort and help from a friend that is more capable, I've managed to extract all 51k game files. I still have no idea how to convert anything into usable formats such as .obj or images and textures that can be viewed. AFAIK this is not possible yet? or am I missing somehting?

Hey man, I tried the only method I can figure out from this forum post. When you say "older method" that you used to get the OBJ files, what method are you specifically referring to and could you give me a rough walk through of how to accomplish this.

Aight I'll break it down Barney Style

 

Step 1: Isolate Files to be processed. Below, I have c/p'd a selection of raw files form the game to a processing folder.

Spoiler

Step 2: Run QuickBMS using this .txt file as the input. Select the .stream/.gpu/.fileless files in your processing folder, and then select a dumping folder (I just use the same processing folder). Now, go get some snacks, take a potty break.

Spoiler

Step 2 will produce a bunch of .mesh and .meshinfo files. Isolate these in their own folder for tracking purposes [I call mine MESHINFO FOLDER] (See my "UNPACKING FOLDER"). Or don't, fuck it, may chaos take the world.

 

Step 3: Download this version of HellExtractor.exe . Place it in your UNPACKING FOLDER (Shown below). Don't use the full release version of HellExtractor, I haven't tested my methods using that one. It probably works better, but fuck it I'm a man of habit. 

Spoiler

Make a .bat called "DRAG MESHINFO FOLDER HERE.bat", using this code
 

Spoiler

@ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION

SET "SCRIPT=%~0"
SET "SCRIPTROOT=%~dp0"

:FILELOOP
    CALL :INDEX "%~1"
    SHIFT /1
    IF "%~n1" == "" (
        PAUSE
        GOTO :EOF
    )
GOTO :FILELOOP

:CHECK
    IF "%~nx1"=="$RECYCLE.BIN" (
        ECHO. > NUL
    ) ELSE IF "%~nx1"=="System Volume Information" (
        ECHO. > NUL
    ) ELSE IF "%~nx1"==".." (
        ECHO. > NUL
    ) ELSE IF "%~nx1"=="." (
        ECHO. > NUL
    ) ELSE IF "%~nx1"=="" (
        ECHO. > NUL
    ) ELSE IF NOT "%~x1"==".meshinfo" (
        ECHO. > NUL
    ) ELSE (
        CALL :INDEX "%~1"
    )
EXIT /B 0

:INDEX
    SET "IS_DIR=0"
    ECHO.%~a1 | find "d" >NUL 2>NUL && (
        set "IS_DIR=1"
    )
    if "!IS_DIR!"=="1" (
        FOR %%i IN ("%~1\*") DO (
            CALL :CHECK "%%~i"
        )
        FOR /D %%i IN ("%~1\*") DO (
            CALL :CHECK "%%~i"
        )
    ) ELSE (
        CALL :TRANSCODE "%~1"
    )
EXIT /B 0

:TRANSCODE
    ECHO %~dpn1
    if NOT EXIST "%~dp1/%~n1" ( mkdir "%~dp1/%~n1" )
    IF EXIST "%SCRIPTROOT%\Debug\hellextractor.exe" (
        "%SCRIPTROOT%\Debug\hellextractor.exe" "%~1" > "%~dp1/%~n1/log.log"
    ) ELSE IF EXIST "%SCRIPTROOT%\RelWithDebInfo\hellextractor.exe" (
        "%SCRIPTROOT%\RelWithDebInfo\hellextractor.exe" "%~1" > "%~dp1/%~n1/log.log"
    ) ELSE IF EXIST "%SCRIPTROOT%\hellextractor.exe" (
        "%SCRIPTROOT%\hellextractor.exe" "%~1" > "%~dp1/%~n1/log.log"
    )
    if ERRORLEVEL 1 (
        ECHO "An unknown error occured."
        PAUSE
    )
EXIT /B 0

Man I sure hope that pasted right. If not, look here

Drag your MESHINFO FOLDER to the .bat, while the HellExtractor.exe/.pdb is in the same UNPACKING FOLDER. Again, go get some snacks, take a potty break.

 

Step 4: Congrats, you should now have a folder full of folders filled with .obj's. You can open the folders individually, or you can be lazy and use this .bat code to extract all the .obj's and put them into a folder where you can organize them by size. Place this .bat in the same MESHINFO FOLDER that all the folders were generated in. Replace the xxx's in the third line with the correct path to your MESHINFO FOLDER. This .bat will delete all folders in MESHINFO FOLDER, DO NOT run this using any other path, it will delete everything it is pointing to.

Spoiler

 

@echo off
setlocal enabledelayedexpansion
set "mainFolder=D:\xxx\xxx\xxx\MESHINFO FOLDER"
set "counter=0"

REM Create the "Extracted OBJs" subfolder
mkdir "%mainFolder%\Extracted OBJs" 2>nul

REM Create the "meshes-meshinfo" subfolder
mkdir "%mainFolder%\meshes-meshinfo" 2>nul

for /d %%i in ("%mainFolder%\*") do (
    for %%j in ("%%i\*.obj") do (
        set "newName=!counter!_%%~nxj"
        move "%%j" "%mainFolder%\Extracted OBJs\!newName!"
        set /a counter+=1
    )
    rd "%%i"
)

REM Move all files in the main folder (except UNPACKER.bat) to "meshes-meshinfo" subfolder
for %%f in ("%mainFolder%\*") do (
    if not "%%~nxf"=="UNPACKER.bat" (
        move "%%f" "%mainFolder%\meshes-meshinfo\" 2>nul
    )
)

REM Delete all directories in the main folder except for "Extracted OBJs" and "meshes-meshinfo"
for /d %%d in ("%mainFolder%\*") do (
    if /i not "%%~nxd"=="Extracted OBJs" (
        if /i not "%%~nxd"=="meshes-meshinfo" (
            rd /s /q "%%d"
        )
    )
)

 

Technically you could do this whole shebang for all game files all at once, but I take it a piece at a time (~10 GB sections of original game files). This is not the most efficient means of doing this, Xaymar's finished product probably handles this better but like I said earlier, fuck it, this works for me.

 

Hope this helps. If anyone's got input or a better way of doing this, please roast me.

Edited March 13 by indy_
clarity

[tugoose]

On 3/14/2024 at 6:56 AM, indy_ said:

Aight I'll break it down Barney Style

 

Step 1: Isolate Files to be processed. Below, I have c/p'd a selection of raw files form the game to a processing folder.

  Hide contents

Step 2: Run QuickBMS using this .txt file as the input. Select the .stream/.gpu/.fileless files in your processing folder, and then select a dumping folder (I just use the same processing folder). Now, go get some snacks, take a potty break.

  Hide contents

Step 2 will produce a bunch of .mesh and .meshinfo files. Isolate these in their own folder for tracking purposes [I call mine MESHINFO FOLDER] (See my "UNPACKING FOLDER"). Or don't, fuck it, may chaos take the world.

 

Step 3: Download this version of HellExtractor.exe . Place it in your UNPACKING FOLDER (Shown below). Don't use the full release version of HellExtractor, I haven't tested my methods using that one. It probably works better, but fuck it I'm a man of habit. 

  Reveal hidden contents

Make a .bat called "DRAG MESHINFO FOLDER HERE.bat", using this code
 

  Reveal hidden contents

@ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION

SET "SCRIPT=%~0"
SET "SCRIPTROOT=%~dp0"

:FILELOOP
    CALL :INDEX "%~1"
    SHIFT /1
    IF "%~n1" == "" (
        PAUSE
        GOTO :EOF
    )
GOTO :FILELOOP

:CHECK
    IF "%~nx1"=="$RECYCLE.BIN" (
        ECHO. > NUL
    ) ELSE IF "%~nx1"=="System Volume Information" (
        ECHO. > NUL
    ) ELSE IF "%~nx1"==".." (
        ECHO. > NUL
    ) ELSE IF "%~nx1"=="." (
        ECHO. > NUL
    ) ELSE IF "%~nx1"=="" (
        ECHO. > NUL
    ) ELSE IF NOT "%~x1"==".meshinfo" (
        ECHO. > NUL
    ) ELSE (
        CALL :INDEX "%~1"
    )
EXIT /B 0

:INDEX
    SET "IS_DIR=0"
    ECHO.%~a1 | find "d" >NUL 2>NUL && (
        set "IS_DIR=1"
    )
    if "!IS_DIR!"=="1" (
        FOR %%i IN ("%~1\*") DO (
            CALL :CHECK "%%~i"
        )
        FOR /D %%i IN ("%~1\*") DO (
            CALL :CHECK "%%~i"
        )
    ) ELSE (
        CALL :TRANSCODE "%~1"
    )
EXIT /B 0

:TRANSCODE
    ECHO %~dpn1
    if NOT EXIST "%~dp1/%~n1" ( mkdir "%~dp1/%~n1" )
    IF EXIST "%SCRIPTROOT%\Debug\hellextractor.exe" (
        "%SCRIPTROOT%\Debug\hellextractor.exe" "%~1" > "%~dp1/%~n1/log.log"
    ) ELSE IF EXIST "%SCRIPTROOT%\RelWithDebInfo\hellextractor.exe" (
        "%SCRIPTROOT%\RelWithDebInfo\hellextractor.exe" "%~1" > "%~dp1/%~n1/log.log"
    ) ELSE IF EXIST "%SCRIPTROOT%\hellextractor.exe" (
        "%SCRIPTROOT%\hellextractor.exe" "%~1" > "%~dp1/%~n1/log.log"
    )
    if ERRORLEVEL 1 (
        ECHO "An unknown error occured."
        PAUSE
    )
EXIT /B 0

Man I sure hope that pasted right. If not, look here

Drag your MESHINFO FOLDER to the .bat, while the HellExtractor.exe/.pdb is in the same UNPACKING FOLDER. Again, go get some snacks, take a potty break.

 

Step 4: Congrats, you should now have a folder full of folders filled with .obj's. You can open the folders individually, or you can be lazy and use this .bat code to extract all the .obj's and put them into a folder where you can organize them by size. Place this .bat in the same MESHINFO FOLDER that all the folders were generated in. Replace the xxx's in the third line with the correct path to your MESHINFO FOLDER. This .bat will delete all folders in MESHINFO FOLDER, DO NOT run this using any other path, it will delete everything it is pointing to.

  Reveal hidden contents

 

@echo off
setlocal enabledelayedexpansion
set "mainFolder=D:\xxx\xxx\xxx\MESHINFO FOLDER"
set "counter=0"

REM Create the "Extracted OBJs" subfolder
mkdir "%mainFolder%\Extracted OBJs" 2>nul

REM Create the "meshes-meshinfo" subfolder
mkdir "%mainFolder%\meshes-meshinfo" 2>nul

for /d %%i in ("%mainFolder%\*") do (
    for %%j in ("%%i\*.obj") do (
        set "newName=!counter!_%%~nxj"
        move "%%j" "%mainFolder%\Extracted OBJs\!newName!"
        set /a counter+=1
    )
    rd "%%i"
)

REM Move all files in the main folder (except UNPACKER.bat) to "meshes-meshinfo" subfolder
for %%f in ("%mainFolder%\*") do (
    if not "%%~nxf"=="UNPACKER.bat" (
        move "%%f" "%mainFolder%\meshes-meshinfo\" 2>nul
    )
)

REM Delete all directories in the main folder except for "Extracted OBJs" and "meshes-meshinfo"
for /d %%d in ("%mainFolder%\*") do (
    if /i not "%%~nxd"=="Extracted OBJs" (
        if /i not "%%~nxd"=="meshes-meshinfo" (
            rd /s /q "%%d"
        )
    )
)

 

Technically you could do this whole shebang for all game files all at once, but I take it a piece at a time (~10 GB sections of original game files). This is not the most efficient means of doing this, Xaymar's finished product probably handles this better but like I said earlier, fuck it, this works for me.

 

Hope this helps. If anyone's got input or a better way of doing this, please roast me.

Okay appreciate the reply, ngl this requires some decoding and on my end to understand this fully but with some mental effort this should hopefully prove very useful. I have realised that I have ".wwise_stream" files instead of ".stream". I assume this may be related to the hellextractor version im using? I'll try with these and then maybe attempt the downgraded version to see if that helps.

 

Very much appreciate you taking the time to write this either way. Have a wonderful day, friend.

[Glor]

You could also change script a little so quickbms wouldn't terminate on first file with no stream/gpu_resource file association

Open FDDE stream 1 stream_exists
Open FDDE gpu_resources 2 gpu_exists

if stream_exists == 0
  Exit
endif

if gpu_exists == 0
  Exit
endif

 

Edited March 17 by Glor
typoes

[xypwn]

Hi everyone!

I've initially found this thread because I was interested in how HD2 stores its data, and well... this lead me down quite a rabbit hole.

I had basically no experience in reverse engineering before this, but decent bit of experience in programming. So I decided to write a data mining tool based on the work of people in this thread, as well as some other people.

Here's what I came up with: https://github.com/xypwn/filediver

So far, I've got working:

- Audio (I basically reimplemented a lot of ww2ogg in Go, so my program supports wwise vorbis natively, no CGo)

- Textures and video (currently just directly ripping binary data from the stream/GPU files, with FFmpeg converting the videos)

Currently I'm trying to parse/understand Stingray's "unit" 3D file format. I'm having a bit of a hard time though, being that I don't really have any experience with any kind of 3D format. So if someone happened to have something like a comprehensive description of the "unit" format, that would be fantastic.

Anyways, thanks to everyone on this thread, this has been a great learning experience!

EDIT: Got models working, and after some reverse engineering, even some basic materials.

Edited March 21 by xypwn
update

[terminid]

Hi All!,

Just wanted to mention DTZxPorters Diver tool: https://dtzxporter.com/tools/diver. I've been able to use it to get at models, animations, textures and audio files!

Keep on making great things!!

[kidi]

hello! i hope asking this here is not off-topic, but i was wondering if there is a way to import the extracted files back into the game? I'm trying to replace the game music and I don't really have much experience replacing assets so i was hoping maybe someone already figure it out

[ThunderRunner]

Does anyone have a list of the armor files? I'm trying to learn how to do the mesh replacements like what was done for the master chief and clone trooper mods, and have no idea where to start.