Helldivers 2 Model Extraction Help!

Xaymar · February 23

Thanks, that was what was missing for the other meshes.

This kind of made it clear to me that the thing I considered the LOD group is not actually the LOD group. It is simply a descriptor for where and how the mesh information is stored. So there's something else that actually binds these meshes into one LOD.

Edit: Full code is at https://github.com/Xaymar/Sandbox-Cpp/tree/master/boxes/hellextractor, in case someone else also wants to have a shot at it.

Edit 2: After a few hours of testing, I found more things that don't quite work right.

There are data types with vertex strides of 24, 32 and 40, which just have 4 bytes of 0xFF padding inserted at the start.
- The unusual thing about these is that any Mesh that uses these data types ends up having insanely large index/vertex counts and offsets, far beyond the actual file. See example 1 below.
Have not yet found a single file with 32bit indices, only 16bit so far.
Still no idea what the rest of the vertex data is, doesn't appear to be normals. Tangents maybe?
UV maps are kind of tiny, and I don't know why yet.

# Example 1
build\boxes\hellextractor\Debug\hellextractor.exe D:\output\000d250a449ec1e8_meshes\0x4A2D0C5CD862E9EA.meshinfo D:\output\000d250a449ec1e8_meshes\0x4A2D0C5CD862E9EA.mesh
...
mesh_0045.obj
  Type: 24
  Indices: 474144/276
  Vertices: 215575/165
Vertex list malformed, skipping.
mesh_0046.obj
  Type: 24
  Indices: 469302/276
  Vertices: 213002/165
Vertex list malformed, skipping.
mesh_0047.obj
  Type: 24
  Indices: 461844/276
  Vertices: 209367/165
Vertex list malformed, skipping.
mesh_0048.obj
  Type: 24
  Indices: 451896/276
  Vertices: 205048/165
Vertex list malformed, skipping.

# All of the above are placing the read cursor far outside the actual file.

Edit 3: The 2nd/3rd entries of the HashTable are the 64bit Material hashes, tied to the DataType itself. So it controls how much data is actually present on a vertex, as well as what material will be used.

Spoiler

//------------------------------------------------
//--- 010 Editor v14.0 Binary Template
//
//      File: 
//   Authors: 
//   Version: 
//   Purpose: 
//  Category: 
// File Mask: 
//  ID Bytes: 
//   History: 
//------------------------------------------------
OutputPaneClear();LittleEndian();

local uint32 i,j,k,l,m,n,o;

struct HEADER {
    uint32 __unk00[23];
    uint32 DataTypeOffset;
    uint32 __unk01;
    uint32 MeshMainInfoOffset;
    uint32 __unk02[2];
    uint32 HashTableOffset;
}Header;

FSeek(Header.DataTypeOffset);
struct DataType {
    uint32 Count;
    uint32 Offsets[Count];
    struct {
        for (i=0; i < Count; i++){
            FSeek(startof(DataTypeInfo)+Offsets[i]);
            struct {
                uint32 __unk00[17];
                uint32 __unique00;
                uint32 __unk03[70];
                uint32 VtxCount, VtxStride;
                uint32 __unk01[8];
                uint32 IdxCount;
                uint32 __unk02[5];
                uint32 VtxBlockOffset,VtxBlockSize,IdxBlockOffset,IdxBlockSize;
            }DataType;    
        }    
    }DataTypes;
}DataTypeInfo;

FSeek(Header.MeshMainInfoOffset);
struct MESHMAININFO {
    uint32 MeshCount;
    uint32 Offsets[MeshCount];
    uint32 MeshCRC[MeshCount];
    uint32 Null;
    struct {
        for (j=0; j < MeshMainInfo.MeshCount; j++){
            FSeek(startof(MeshMainInfo.Offsets[0])+MeshMainInfo.Offsets[j]);
            struct MESHINFO {
                uint32 __unk00[8];
                uint32 flags;
                uint32 __unk01[22];
                uint32 DataTypeHash;
                uint32 __unk02;
                uint32 VtxOffset,VtxCount,IdxOffset,IdxCount;
            }Mesh;
        };
    } Meshes;
}MeshMainInfo;

FSeek(Header.HashTableOffset);
struct HashTable {
    uint32 Count;
    uint32 DataTypeHashes[Count];
    uint64 MaterialHashes[Count];
}HashInfo;

Edited February 24 by Xaymar
BMS update

h3x3r · February 23

@Xaymar

Quote

Have not yet found a single file with 32bit indices, only 16bit so far.

This file: 0d48fc8d98505493 has 32bit indices

Also discovered that files without *.gpu_resources are mostly audio/video containers. So there's no point of unpacking it since there is no model/texture.

Edited February 23 by h3x3r

Xaymar · February 23

@h3x3r

Seems like I'm missing that file, will have to figure out how to do the QuickBMS export properly later. Currently struggling to get the 24, 32 and 40 byte wide vertex stride to work right, they all have funny results that aren't correct (see picture). Maybe I'll even be able to decode the whole material file into a proper mtl file for the obj.

Left to right we have: 40 byte, 32 byte and 20 byte. The only one that looks correct here is the 20 byte one, the rest is weirdly distorted. Fun. But considering the UV map looks exactly the same, maybe it's meant to be like that? Or maybe there's some additional information somewhere that tells me if the indices list is points, lines, linestrip, tris, tristrip, quads, quadstrip, etc. Will stumble across it eventually.

h3x3r · February 23

Here is that file... Anyway my template it's just guesswork don't take it seriously.

0d48fc8d98505493_meshes.7z

GameBreaker · February 23

Currently working on a Python script for unpacking the game files thanks to h3x3r's work and some other helpful fellows, as that can then be transferred elsewhere for use, like in Blender.

Edited February 23 by GameBreaker

h3x3r · February 23

OK so i edited bms script and now it should work fine.

######################################################
#  Helldivers 2 - Texture / Mesh / Material  Dumper  #
######################################################
get BaseFileName basename

Open FDDE "" 0

get Magic long
get UnknownCount long
get Files long
getdstring Dummy 0x3C
getdstring Dummy UnknownCount*32

for i = 0 < Files
	endian big
	get FileNameCRC0 long
	get FileNameCRC1 long
	endian little
	get FileTypeCRC64 longlong
	get MainFileOffset long
    get Unknown_0 long
    get StreamFileOffset long
    get Unknown_1 long
    get GpuResourcesFileOffset long
    get Unknown_2 long
    get Unknown_3 long
    get Unknown_4 long
    get Unknown_5 long
    get Unknown_6 long
    get MainFileSize long
    get StreamFileSize long
    get GpuResourcesFileSize long
    get Unknown_7 long
    get Unknown_8 long
    get FileNum long
	savepos EndTable
	
	if FileTypeCRC64 == 14790446551990181426
		if StreamFileSize != 0
			Open FDDE stream 1
			goto MainFileOffset
			getdstring Dummy 0xC0
			savepos MainFileOffset
			string Name p= "%s_textures/0x%04X%04X.dds" BaseFileName FileNameCRC0 FileNameCRC1
			append 0
			log Name MainFileOffset 148 0
			log Name StreamFileOffset StreamFileSize 1
		else
			Open FDDE gpu_resources 2
			goto MainFileOffset
			getdstring Dummy 0xC0
			savepos MainFileOffset
			string Name p= "%s_textures/0x%04X%04X.dds" BaseFileName FileNameCRC0 FileNameCRC1
			append 0
			log Name MainFileOffset 148 0
			log Name GpuResourcesFileOffset GpuResourcesFileSize 2
		endif
		
	elif FileTypeCRC64 == 16187218042980615487
		if GpuResourcesFileSize != 0 && MainFileSize != 0
			Open FDDE gpu_resources 2
			string MdlInfoName p= "%s_meshes/0x%04X%04X.meshinfo" BaseFileName FileNameCRC0 FileNameCRC1
			string MdlDataName p= "%s_meshes/0x%04X%04X.mesh" BaseFileName FileNameCRC0 FileNameCRC1
			log MdlInfoName MainFileOffset MainFileSize 0
			log MdlDataName GpuResourcesFileOffset GpuResourcesFileSize 2
		endif
		
	elif FileTypeCRC64 == 16915718763308572383
		if MainFileSize != 0
			string MatInfoName p= "%s_meshes/mat/0x%04X%04X.mat" BaseFileName FileNameCRC0 FileNameCRC1
			log MatInfoName MainFileOffset MainFileSize 0
		endif
	endif
	goto EndTable
next i

Xaymar · February 23

Thanks for the new/updated BMS script, it actually managed to export most if not all content for me this time. With it I was able to figure out some additional information, such as that meshes can have multiple materials, the hashtable at the end is a map<Hash, MaterialFile> map, and some other things. See updated 010.

Spoiler

//------------------------------------------------
//--- 010 Editor v14.0 Binary Template
//
//      File: 
//   Authors: 
//   Version: 
//   Purpose: 
//  Category: 
// File Mask: 
//  ID Bytes: 
//   History: 
//------------------------------------------------
OutputPaneClear();LittleEndian();

local uint32 i,j,k,l,m,n,o;

struct Vector2 {
    float x, y;
};

struct Vector3 {
    float x, y, z;
};

struct Vector4 {
    float x, y, z, w;
};

struct Matrix3x4{
    Vector3 x, y, z, w;
};

struct Matrix4x4{
    Vector4 x, y, z, w;
};

struct HEADER {
    uint32 __unk00[12];
    uint32 Unknown3Offset;
    uint32 Unknown4Offset;
    uint32 Unknown5Offset;
    uint32 Unknown6Offset;
    uint32 __unk01[3];
    uint32 Unknown1Offset;
    uint32 Unknown2Offset;
    uint32 __unk02;
    uint32 UnknownOffset;
    uint32 DataTypeOffset;
    uint32 Unknown7Offset;;
    uint32 MeshInfoOffset;
    uint32 __unk04[2];
    uint32 MaterialHashMapOffset;
}Header;

if(Header.Unknown6Offset > 0) {
    FSeek(Header.Unknown6Offset);
    struct Unknown6 {
        char __unk00[16];
    }Unknown6Info;
}

if(Header.Unknown5Offset > 0) {
    FSeek(Header.Unknown5Offset);
    struct Unknown5 {
        uint32 Count;
        char __unk00[12];
        struct {
            char __unk00[8];
            float textureWidth, textureHeight, textureDepth;
            char __unk01[92];
        }Unknown5Data[Count];
    }Unknown5Info;
}

if(Header.Unknown3Offset > 0) {
    FSeek(Header.Unknown3Offset);
    struct Unknown3 {
        uint32 Count;
        struct{
            char __unk00[159];
        }Unknown3Data[Count];
    }Unknown3Info;
}

if(Header.Unknown4Offset > 0) {
    FSeek(Header.Unknown4Offset);
    struct Unknown4 {
        uint32 Count;
        struct {
            uint32 __unk00[34];
        }Unknown4Data[Count];
        // 22192 bytes total?, 56B0
    }Unknown4Info;
}

if(Header.Unknown1Offset > 0) {
    FSeek(Header.Unknown1Offset);
    struct Unknown1 {
        uint32 Count;
        uint32 __unk00;
    }Unknown1Info;
}

if(Header.Unknown2Offset > 0) {
    FSeek(Header.Unknown2Offset);
    struct Unknown2 {
        struct {
            uint32 __unk00[8];
        }Unknown2Data[3];
    }Unknown2Info;
}

if(Header.UnknownOffset > 0) {
    FSeek(Header.UnknownOffset);
    struct Unknown {
        uint32 count;
        uint32 offsets[count];
        struct {
            for (i=0; i < count; i++){
                FSeek(startof(UnknownInfo)+offsets[i]);
                struct {
                    uint32 __unk00[17];
                }UnknownData;
            }    
        }UnknownDatas;
    }UnknownInfo;
}

if(Header.DataTypeOffset > 0) {
    FSeek(Header.DataTypeOffset);
    struct dataTypeInfo {
        uint32 count;
        uint32 offsets[count];
        uint32 __identicalAcrossTypes[count];
        uint32 __unk00;
        struct dataTypes {
            for (i=0; i < count; i++){
                FSeek(startof(DataTypeInfo)+offsets[i]);
                struct dataType{
                    uint32 __unk00[2];
                    uint32 __varies00[2];
                    uint32 __unk01[3];
                    uint32 __varies01[2];
                    uint32 __unk02[3];
                    uint32 __varies02[2];
                    uint32 __unk03[2];
                    uint32 __unk04;
                    uint32 __varies03[3];
                    uint32 __unk05[2];
                    uint32 __varies04[2];
                    uint32 __unk06[3];
                    uint32 __varies05[2];
                    uint32 __unk07[3];
                    uint32 __unk08[56];
                    uint32 vertexCount;
                    uint32 vertexStride;
                    uint32 __unk09[8];
                    uint32 indexCount;
                    uint32 __unk10[5];
                    uint32 vertexOffset;
                    uint32 vertexSize;
                    uint32 indexOffset;
                    uint32 indexSize;
                    uint32 __unk11[4];
                }DataType;
            }    
        }DataTypes;
    }DataTypeInfo;
}

if(Header.MeshInfoOffset > 0) {
    FSeek(Header.MeshInfoOffset);
    struct {
        uint32 count;
        uint32 offsets[count];
        uint32 __identicalAcrossTypes[count];
        struct {
            for (j=0; j < MeshInfo.count; j++){
                FSeek(startof(MeshInfo.offsets[0])+MeshInfo.offsets[j]);
                struct {
                    uint32 __unk00;
                    uint32 __varies00[7];
                    uint32 __unk01;
                    uint32 __sameAsIndenticalAcrossTypesAtIdx;
                    uint32 __varies01[2];
                    uint32 __unk02;
                    uint32 __varies02;
                    uint32 dataTypeIndex;
                    uint32 __unk03[1];
                    uint32 __unk04[9];
                    uint32 materialCount2;
                    uint32 __unk05[3];
                    uint32 materialCount;
                    uint32 vertexInfoOffset;
                    uint32 materialHashes[materialCount];
                    uint32 __unk06;
                    FSeek(startof(Mesh[j])+vertexInfoOffset);
                    uint32 vertexOffset;
                    uint32 vertexCount;
                    uint32 indexOffset;
                    uint32 indexCount;
                    
                    //dataTypeInfo.dataTypes.dataType DataTypeInfo.DataTypes[dataTypeIndex];
                }Mesh;
            };
        } Meshes;
    }MeshInfo;
}

if(Header.MaterialHashMapOffset > 0) {
    FSeek(Header.MaterialHashMapOffset);
    struct {
        uint32 Count;
        uint32 DataTypeHashes[Count];
        uint64 MaterialHash[Count];
    }MaterialHashMap;
}

if(Header.Unknown7Offset > 0) {
    FSeek(Header.Unknown7Offset);
    struct Unknown7 {
        uint32 __unk00[2];
    }Unknown7Info;
}

The list of Key,Value hashes at the end of the file are for mapping some sort of Key (stored as a 32bit hash) to the full Material (stored as a 64bit hash).
vertexOffset, vertexCount, indexOffset, and indexCount are located at MeshInfo+vertexInfoOffset.
Once again back to no idea how to map meshes to their data type. Perhaps it cycles through since meshes seem to be listed in backwards order towards offset 0.

Edit:

Vertex Indices can be 8bit, 16bit or 32bit so far.
DataType List has a CRC or similar after the offsets. This list of CRCs can also be found again at a much earlier location in the file.

Edit 2:

The "CRC" entries are not CRCs, and repeat quite often. Seems like a fixed list of numbers specific to a file.

Edit 3:

I think i found it. There's a value after the weird value that usually is always 0xFFFFFFFF that seems to count randomly, but doesn't appear to go higher than the maximum number of datatypes.

Edit 4:

The index stride can be inferred by dividing index_size by index_count. This will result in either 1, 2, 4, or 8. Meshes have no say over this is seems.
What's left to figure out is how these models are grouped into one LOD, as clearly some of them are a lower quality model of another. It's unlikely that this information is hardcoded into the game executable, so there's one more thing to figure out.

Edited February 24 by Xaymar

GameBreaker · February 24

Here is my edit to include a file check for the QBMS script:
You might have to run QBMS 4 GB in admin to run this as I did.
```

######################################################
# Helldivers 2 - Texture / Mesh / Material Dumper #
######################################################
get BaseFileName basename

Open FDDE stream 1 streamEXISTS
Open FDDE gpu_resources 2 gpu_resourceEXISTS

get Magic long
get UnknownCount long
get Files long
getdstring Dummy 0x3C
getdstring Dummy UnknownCount*32

for i = 0 < Files
   endian big
   get FileNameCRC0 long
   get FileNameCRC1 long
   endian little
   get FileTypeCRC64 longlong
   get MainFileOffset long
    get Unknown_0 long
    get StreamFileOffset long
    get Unknown_1 long
    get GpuResourcesFileOffset long
    get Unknown_2 long
    get Unknown_3 long
    get Unknown_4 long
    get Unknown_5 long
    get Unknown_6 long
    get MainFileSize long
    get StreamFileSize long
    get GpuResourcesFileSize long
    get Unknown_7 long
    get Unknown_8 long
    get FileNum long
   savepos EndTable

   if FileTypeCRC64 == 14790446551990181426
       if StreamFileSize != 0 & streamEXISTS = 1
           goto MainFileOffset
           getdstring Dummy 0xC0
           savepos MainFileOffset
           string Name p= "%s_textures/0x%04X%04X.dds" BaseFileName FileNameCRC0 FileNameCRC1
           append 0
           log Name MainFileOffset 148 0
           log Name StreamFileOffset StreamFileSize 1
       elif GpuResourcesFileSize != 0 & gpu_resourceEXISTS = 1
           goto MainFileOffset
           getdstring Dummy 0xC0
           savepos MainFileOffset
           string Name p= "%s_textures/0x%04X%04X.dds" BaseFileName FileNameCRC0 FileNameCRC1
           append 0
           log Name MainFileOffset 148 0
           log Name GpuResourcesFileOffset GpuResourcesFileSize 2
       endif

   elif FileTypeCRC64 == 16187218042980615487
       if GpuResourcesFileSize != 0 & MainFileSize != 0 & gpu_resourceEXISTS = 1
           string MdlInfoName p= "%s_meshes/0x%04X%04X.meshinfo" BaseFileName FileNameCRC0 FileNameCRC1
           string MdlDataName p= "%s_meshes/0x%04X%04X.mesh" BaseFileName FileNameCRC0 FileNameCRC1
           log MdlInfoName MainFileOffset MainFileSize
           log MdlDataName GpuResourcesFileOffset GpuResourcesFileSize 2
       endif
   elif FileTypeCRC64 == 16915718763308572383
       if MainFileSize != 0
           string MatInfoName p= "%s_meshes/mat/0x%04X%04X.mat" BaseFileName FileNameCRC0 FileNameCRC1
           log MatInfoName MainFileOffset MainFileSize
       endif
   endif
   goto EndTable
next i
```

Xaymar · February 24

Thanks to the new BMS scripts, I got exporting of files working almost perfectly now:

Remaining questions:

vertex20_t:
- What is offset 0x0C for? Could fit 4 bytes, 2 half/shorts, or 1 float. Can reasonably rule out halfs, since some of the values are NaN.
vertex24_t:
- Same questions as vertex20_t, but with +0x4 to all offsets.
- What is offset 0x00 for?
vertex28_t:
- Same questions as vertex20_t.
- What is offset 0x14 for?
- What is offset 0x18 for?
vertex32_t:
- Same questions as vertex28_t, but with +0x4 to all offsets.
vertex36_t:
- Same questions as vertex28_t.
- What is offset 0x1C for?
- What is offset 0x20 for?
vertex40_t:
- Same questions as vertex36_t, but with +0x4 to all offsets.
If the meshes have multiple UV maps, what model format could I switch to that is easy to create, yet supports that?
- Wavefront OBJ is limited to a single UV layer per group.
Where is the "mesh name" that would group the whole set into one?
- Where is the relative LOD index to the "mesh name" group?

Code on github is up-to-date again, and below is the latest 010 file that I used to figure all this out.

Spoiler

//------------------------------------------------
//--- 010 Editor v14.0 Binary Template
//
//      File: 
//   Authors: 
//   Version: 
//   Purpose: 
//  Category: 
// File Mask: 
//  ID Bytes: 
//   History: 
//------------------------------------------------
OutputPaneClear();LittleEndian();

typedef uint64 uint64_t;
typedef uint32 uint32_t;
typedef uint16 uint16_t;
typedef char uint8_t;
typedef uint16_t half;

local uint32 i,j,k,l,m,n,o;

struct Vector2 {
    float x, y;
};

struct Vector3 {
    float x, y, z;
};

struct Vector4 {
    float x, y, z, w;
};

struct Matrix3x4{
    Vector3 x, y, z, w;
};

struct Matrix4x4{
    Vector4 x, y, z, w;
};

struct HEADER {
    uint32 __unk00[12];
    uint32 Unknown3Offset;
    uint32 Unknown4Offset;
    uint32 Unknown5Offset;
    uint32 Unknown6Offset;
    uint32 __unk01[3];
    uint32 Unknown1Offset;
    uint32 Unknown2Offset;
    uint32 __unk02;
    uint32 UnknownOffset;
    uint32 DataTypeOffset;
    uint32 Unknown7Offset;;
    uint32 MeshInfoOffset;
    uint32 __unk04[2];
    uint32 MaterialHashMapOffset;
}Header;

if(Header.Unknown6Offset > 0) {
    FSeek(Header.Unknown6Offset);
    struct Unknown6 {
        char __unk00[16];
    }Unknown6Info;
}

if(Header.Unknown5Offset > 0) {
    FSeek(Header.Unknown5Offset);
    struct Unknown5 {
        uint32 Count;
        char __unk00[12];
        struct {
            char __unk00[8];
            float textureWidth, textureHeight, textureDepth;
            char __unk01[92];
        }Unknown5Data[Count];
    }Unknown5Info;
}

if(Header.Unknown3Offset > 0) {
    FSeek(Header.Unknown3Offset);
    struct Unknown3 {
        uint32 Count;
        struct{
            char __unk00[159];
        }Unknown3Data[Count];
    }Unknown3Info;
}

if(Header.Unknown4Offset > 0) {
    FSeek(Header.Unknown4Offset);
    struct Unknown4 {
        uint32 Count;
        struct {
            uint32 __unk00[34];
        }Unknown4Data[Count];
        // 22192 bytes total?, 56B0
    }Unknown4Info;
}

if(Header.Unknown1Offset > 0) {
    FSeek(Header.Unknown1Offset);
    struct Unknown1 {
        uint32 Count;
        uint32 __unk00;
    }Unknown1Info;
}

if(Header.Unknown2Offset > 0) {
    FSeek(Header.Unknown2Offset);
    struct Unknown2 {
        struct {
            uint32 __unk00[8];
        }Unknown2Data[3];
    }Unknown2Info;
}

if(Header.UnknownOffset > 0) {
    FSeek(Header.UnknownOffset);
    struct Unknown {
        uint32 count;
        uint32 offsets[count];
        struct {
            for (i=0; i < count; i++){
                FSeek(startof(UnknownInfo)+offsets[i]);
                struct {
                    uint32 __unk00[17];
                }UnknownData;
            }    
        }UnknownDatas;
    }UnknownInfo;
}

if(Header.DataTypeOffset > 0) {
    FSeek(Header.DataTypeOffset);
    struct dataTypeInfo {
        uint32 count;
        uint32 offsets[count];
        uint32 __identicalAcrossTypes[count];
        uint32 __unk00;
        struct dataTypes {
            for (i=0; i < count; i++){
                FSeek(startof(DataTypeInfo)+offsets[i]);
                struct dataType{
                    uint32 __unk00[2];
                    uint32 __varies00[2];
                    uint32 __unk01[3];
                    uint32 __varies01[2];
                    uint32 __unk02[3];
                    uint32 __varies02[2];
                    uint32 __unk03[2];
                    uint32 __unk04;
                    uint32 __varies03[3];
                    uint32 __unk05[2];
                    uint32 __varies04[2];
                    uint32 __unk06[3];
                    uint32 __varies05[2];
                    uint32 __unk07[3];
                    uint32 __unk08[56];
                    uint32 vertex_count;
                    uint32 vertex_stride;
                    uint32 __unk09[8];
                    uint32 index_count;
                    uint32 __unk10[5];
                    uint32 vertex_offset;
                    uint32 vertex_size;
                    uint32 index_offset;
                    uint32 index_size;
                    uint32 __unk11[4];
                }DataType;
            }    
        }DataTypes;
    }DataTypeInfo;
}

if(Header.MeshInfoOffset > 0) {
    FSeek(Header.MeshInfoOffset);
    struct {
        uint32 count;
        uint32 offsets[count];
        uint32 __identicalAcrossTypes[count];
        struct {
            for (j=0; j < MeshInfo.count; j++){
                FSeek(startof(MeshInfo.offsets[0])+MeshInfo.offsets[j]);
                struct {
                    uint32 __unk00;
                    uint32 __varies00[7];
                    uint32 __unk01;
                    uint32 __sameAsIndenticalAcrossTypesAtIdx;
                    uint32 __varies01[2];
                    uint32 __unk02;
                    uint32 __varies02;
                    uint32 dataTypeIndex;
                    uint32 __unk03[10];
                    uint32 material_count2;
                    uint32 __unk04[3];
                    uint32 material_count;
                    uint32 modelinfo_offset;
                    uint32 materials[material_count];
                    uint32 __unk05;
                    FSeek(startof(Mesh[j])+modelinfo_offset);
                    uint32 vertex_offset;
                    uint32 vertex_count;
                    uint32 index_offset;
                    uint32 index_count;
                    
                    //dataTypeInfo.dataTypes.dataType DataTypeInfo.DataTypes[dataTypeIndex];
                }Mesh;
            };
        } Meshes;
    }MeshInfo;
}

if(Header.MaterialHashMapOffset > 0) {
    FSeek(Header.MaterialHashMapOffset);
    struct {
        uint32 Count;
        uint32 DataTypeHashes[Count];
        uint64 MaterialHash[Count];
    }MaterialHashMap;
}

if(Header.Unknown7Offset > 0) {
    FSeek(Header.Unknown7Offset);
    struct Unknown7 {
        uint32 __unk00[2];
    }Unknown7Info;
}

Edit: Some more models that successfully extract now:

Edit 2: Figured out why UVs didn't work right: V is 1..0 instead of 0..1.

Now I have UV mapping, for one of the UV layers. Turns out there's a few more than expected.

Edit 4:

> Stride: 48

> Stride: 56

> Stride: 60

> Stride: 96

> Stride: 112

*thousandyardstare.png*

Edit 5:

Stride60: float x, y, z; uint32 __unk00; half u0, v0; half u1, v1; uint32 __unk01; uint32 __unk02[8];

Edit 6:

Well, this sucks. The vertex stride is not what decides the layout, so there's something else defining the actual layout. There's one 16 and 20 byte layout, 3 different 24 byte layout, 18 different 60 byte layouts, ... yaaay.

Edited February 24 by Xaymar

Rohan_Sailor · February 24

Wow! I came here just to see how to get the ship asset for 3D-Printing. I think brain is going to explode.

Xaymar · February 24

Well after many many hours, I'm now at a wall that is seemingly impossible to break through. There are so many possible vertex data layouts based on the stride sizes i've encountered that nothing really makes sense so far. To my knowledge, a mesh always has X, Y, Z, half[2] UV0. There appears to be the option of additional UV layers, but none of the values in the DataType structure match up with the number of textures. Below is an export of the manually discovered structure, as well as the differences for the datatypes:

Spoiler

Stride 16:
    float x, y, z;
    half u0, v0;

Stride 20: // 0x1A051BC6FD00C1ED -> 0
    float x, y, z;
    uint32 unkown;
    half u0, v0;

Stride 24A: // 0x1AF76587B30D2EA -> 0
    float x, y, z;
    uint32 unkown;
    half u0, v0;
    half u1, v1;

Stride 24B: // 0x1B798EF082DEE463 -> 3
    uint32 unkown;
    float x, y, z;
    half u0, v0;
    half u1, v1;

Datatype 20: // 0x1A051BC6FD00C1ED -> 0
    0x008 00
    0x00C 02
    0x01C 01
    0x020 1A
    0x030 04
    0x034 1D
    0x044 00
    0x048 00
    0x04C 00
    0x148 03
    
Datatype 24A: // 0x1AF76587B30D2EA -> 0
    0x008 00
    0x00C 02
    0x01C 01
    0x020 1A
    0x030 04
    0x034 1D
    0x044 04
    0x048 1D
    0x04C 01
    0x148 04
    
Datatype 24B: // 0x1B798EF082DEE463 -> 3
    0x008 05
    0x00C 04
    0x01C 00
    0x020 02
    0x030 01
    0x034 1A
    0x044 04
    0x048 1D
    0x04C 01
    0x148 04

I've only seen a single variant of stride 16 and 20, but I've already seen two variants for stride 24. 0x04C could be (number of uvs - 1), and 0x148 could be a word offset into the vertex data for the beginning of the uv array. That would leave figuring out where x,y,z is, if this even remotely correct. Unfortunately this is where I am stuck now, and it's probably the last piece of the puzzle needed for a full mesh export.

Update: I've had a bit of sleep, and in true programmer fashion figured out a way to hopefully decode this mess. By dumping all datatypes based on their unique changes, ignoring all the parts that stay the same, I can hopefully figure out what is really what. And it turns out, several of my assumptions are already wrong.

For example, there are actually two different 16b vertex structures, so even those ended up weirdly corrupted at times:

float x, y, z; float/uint32 unknown;
float x, y, z; half uv0[2];

... yay. The difference between style 1 and style 2 is just two bytes in the datatype structure:

0x01C shifts from 1 to 4
0x020 shifts from 0x1A to 0x1D

But at least there is not variant with the unkown being a prefix to the position of the vertex for 16b.

I'll work on getting a sample export to upload with enough data to get information about the datatype itself.

Update 2: I've attached a sample of the unique datatypes, plus a small (up to 128 vertices) long export of the matching vertex data. Hopefully someone can help me figure this out.

datatypes.zip

Update 3: I did not expect to be successful, yet here I am. Seems like the DataType structure also holds information about which elements are where. There is a structure at the start of DataType that encodes this information in a clever way. Will update 010 template as I discover more.

uint32[0x00, 0x02, 0x??, 0x??, 0x??] encodes 'float xyz[3];'
uint32[0x01, 0x1A, 0x??, 0x??, 0x??] encodes a single 4 byte wide element of unknown use.
uint32[0x04, 0x1D, 0x#0, 0x??, 0x??] encodes 'half uv[2];', where #0 is the UV layer.
uint32[0x07, 0x1F, 0x??, 0x??, 0x??] is unknown for now.
uint32[0x06, 0x18, 0x#0, 0x??, 0x??] is unknown for now. #0 increments with multiple statements.
uint32[0x06, 0x14, 0x??, 0x??, 0x??] is unknown for now.

Additionally, offset 0x148 appears to be the total number of elements used.

Update 4: Here's the latest 010 template, based on what I figured out.

scripts.zip

Edited February 25 by Xaymar
Trying to fix page weirdness

Xaymar · February 25

My tool seems to work "mostly fine" now, it does not understand some of the formats it encounters yet, but it can handle the important things. I've attached the current version of the tool as a binary below, or you can build from source code using CMake and your favourite compiler of choice. Anyway, quick list of things it does:

Generates a Wavefront .obj file for each mesh inside a meshinfo file.
- Single X,Y,Z position layer
- Single U[,V[,W]] texcoord layer. Additional layers will be present, but as a comment only.
- Single NX,NY,NZ layer if decodable.
- No vertex colors.
- No animations.
- No bones & weights.
Dumps unknown element types and formats to the .obj file as a comment.
Hasn't crashed once in my full game mesh export, but I've also not verified if all exported data is correct. Definitely more correct than before, that's for sure.

Usage: hellextractor.exe PATH/TO/HASH.meshinfo

Will try to load PATH/TO/HASH.meshinfo and PATH/TO/HASH.mesh, then generate the datatypes export (PATH/datatypes/) as well as the meshes in the file (PATH/TO/HASH/00000000.obj).

I still haven't found a common thing between meshes, so still no idea what or even if there is any LOD grouping information present in the .meshinfo. Could be stored in a completely different file that isn't even exported by the BMS scripts so far. Similarily I've found no real information about bones, weights, animations or similar. They'll have to be in there somewhere, right? If we find a model that should be animated, then there's got to be some unique thing to the meshinfo or mesh file that describes bones and weights.

hellextractor.zip

Edited February 25 by Xaymar

DKDave · February 25

I've noticed also that the 'vertex elements' bit doesn't always seem to have the elements in the correct order (another assumption). For example in file 0x2D596CE030D1C0FF.meshinfo there are 3 separate vertex blocks. For the first vertex block, if we assume that 'element type' 0 and 'number type' 2 is equal to 3 Floats, that is always the first bit of data in each vertex, but not always the first entry in the vertex elements info, and it doesn't seem to store the offset of each element within the vertex stride. So they may have to be sorted into order first to process the element offsets correctly. Obviously it's a weird format as usual.

Edit: There seems to be bone weights and indices in vertex stride 0x24, at least in the file I looked at above) - so element type 6 would be weights and type 7 is bone indices. In the example above, the indices are 4 bytes and the weights are 4 half floats.

Myrkur · February 25

Anyone happen to know the name given for the Charger bug and the Guard Dog Rover (laser variant) and its backpack? so far ive only been able to find structural meshes

Helldiver · February 26

2 hours ago, Myrkur said:

Anyone happen to know the name given for the Charger bug and the Guard Dog Rover (laser variant) and its backpack? so far ive only been able to find structural meshes

32cb44321da1b5a0 rover
ea2b289bb6c14cfe charger

Edited February 26 by Helldiver

Myrkur · February 26

15 minutes ago, Helldiver said:

32cb44321da1b5a0 rover

ea2b289bb6c14cfe charger

Thanks!

Xaymar · February 26

8 hours ago, DKDave said:

I've noticed also that the 'vertex elements' bit doesn't always seem to have the elements in the correct order (another assumption). For example in file 0x2D596CE030D1C0FF.meshinfo there are 3 separate vertex blocks. For the first vertex block, if we assume that 'element type' 0 and 'number type' 2 is equal to 3 Floats, that is always the first bit of data in each vertex, but not always the first entry in the vertex elements info, and it doesn't seem to store the offset of each element within the vertex stride. So they may have to be sorted into order first to process the element offsets correctly. Obviously it's a weird format as usual.

Edit: There seems to be bone weights and indices in vertex stride 0x24, at least in the file I looked at above) - so element type 6 would be weights and type 7 is bone indices. In the example above, the indices are 4 bytes and the weights are 4 half floats.

Regarding the first half, the vertex position does not appear to always be the first element. There are a lot of models which have type 0x5 first, which is 4 bytes wide. 0x5 seems to be vertex color.

Latter half, type 7 is a bit weird. It seems to be very normals in half/binary16 precision. it's also usually found without an accompanying type 6 element. I hope that the Stingray engine doesn't use the same type for multiple things, that'd make things complicated 😕

will delve into the file you mentioned.

indy_ · February 26

21 hours ago, Xaymar said:

My tool seems to work "mostly fine" now, it does not understand some of the formats it encounters yet, but it can handle the important things. I've attached the current version of the tool as a binary below, or you can build from source code using CMake and your favourite compiler of choice. Anyway, quick list of things it does:

Generates a Wavefront .obj file for each mesh inside a meshinfo file.

Single X,Y,Z position layer

Single U[,V[,W]] texcoord layer. Additional layers will be present, but as a comment only.

Single NX,NY,NZ layer if decodable.

No vertex colors.

No animations.

No bones & weights.

Dumps unknown element types and formats to the .obj file as a comment.

Hasn't crashed once in my full game mesh export, but I've also not verified if all exported data is correct. Definitely more correct than before, that's for sure.

Usage: hellextractor.exe PATH/TO/HASH.meshinfo

Will try to load PATH/TO/HASH.meshinfo and PATH/TO/HASH.mesh, then generate the datatypes export (PATH/datatypes/) as well as the meshes in the file (PATH/TO/HASH/00000000.obj).

I still haven't found a common thing between meshes, so still no idea what or even if there is any LOD grouping information present in the .meshinfo. Could be stored in a completely different file that isn't even exported by the BMS scripts so far. Similarily I've found no real information about bones, weights, animations or similar. They'll have to be in there somewhere, right? If we find a model that should be animated, then there's got to be some unique thing to the meshinfo or mesh file that describes bones and weights.

hellextractor.zip 523.17 kB · 18 downloads

Works great! Using your extractor in combo with h3x3r's work I was able to pull quite a few .obj's. Found some great stuff, I'm gonna be digging through the files over the next couple of days. Favorite examples in the Spoiler VVV. Think I might write a quick script to automate the process of QuickBMS->meshfiles->HellExtractor->Output, but I've been enjoying doing it by hand and seeing what's in there piece by piece.

Spoiler

Edited February 26 by indy_

Myrkur · February 26

4 hours ago, Helldiver said:

32cb44321da1b5a0 rover

ea2b289bb6c14cfe charger

you happen to also know where the regular dog (gun variant) is, and the hellpods are?

Xaymar · February 26

3 hours ago, indy_ said:

Works great! Using your extractor in combo with h3x3r's work I was able to pull quite a few .obj's. Found some great stuff, I'm gonna be digging through the files over the next couple of days. Favorite examples in the Spoiler VVV. Think I might write a quick script to automate the process of QuickBMS->meshfiles->HellExtractor->Output, but I've been enjoying doing it by hand and seeing what's in there piece by piece.

Reveal hidden contents

Spoiler

@ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION

SET "SCRIPT=%~0"
SET "SCRIPTROOT=%~dp0"

:FILELOOP
	CALL :INDEX "%~1"
	SHIFT /1
	IF "%~n1" == "" (
		PAUSE
		GOTO :EOF
	)
GOTO :FILELOOP

:CHECK
	IF "%~nx1"=="$RECYCLE.BIN" (
		ECHO. > NUL
	) ELSE IF "%~nx1"=="System Volume Information" (
		ECHO. > NUL
	) ELSE IF "%~nx1"==".." (
		ECHO. > NUL
	) ELSE IF "%~nx1"=="." (
		ECHO. > NUL
	) ELSE IF "%~nx1"=="" (
		ECHO. > NUL
	) ELSE IF NOT "%~x1"==".meshinfo" (
		ECHO. > NUL
	) ELSE (
		CALL :INDEX "%~1"
	)
EXIT /B 0

:INDEX
	SET "IS_DIR=0"
	ECHO.%~a1 | find "d" >NUL 2>NUL && (
		set "IS_DIR=1"
	)
	if "!IS_DIR!"=="1" (
		FOR %%i IN ("%~1\*") DO (
			CALL :CHECK "%%~i"
		)
		FOR /D %%i IN ("%~1\*") DO (
			CALL :CHECK "%%~i"
		)
	) ELSE (
		CALL :TRANSCODE "%~1"
	)
EXIT /B 0

:TRANSCODE
	ECHO %~dpn1
	if NOT EXIST "%~dp1/%~n1" ( mkdir "%~dp1/%~n1" )
	IF EXIST "%SCRIPTROOT%\Debug\hellextractor.exe" (
		"%SCRIPTROOT%\Debug\hellextractor.exe" "%~1" > "%~dp1/%~n1/log.log"
	) ELSE IF EXIST "%SCRIPTROOT%\RelWithDebInfo\hellextractor.exe" (
		"%SCRIPTROOT%\RelWithDebInfo\hellextractor.exe" "%~1" > "%~dp1/%~n1/log.log"
	) ELSE IF EXIST "%SCRIPTROOT%\hellextractor.exe" (
		"%SCRIPTROOT%\hellextractor.exe" "%~1" > "%~dp1/%~n1/log.log"
	)
	if ERRORLEVEL 1 (
		ECHO "An unknown error occured."
		PAUSE
	)
EXIT /B 0

Paste the spoilered .bat script into a .bat file next to hellextractor.exe, and then drag the folder with the exported .meshinfo files onto it. Then go get snacks for a bit, on my machine this takes almost 30 minutes to complete.

Edit:

13 hours ago, DKDave said:

Edit: There seems to be bone weights and indices in vertex stride 0x24, at least in the file I looked at above) - so element type 6 would be weights and type 7 is bone indices. In the example above, the indices are 4 bytes and the weights are 4 half floats.

Seems to be flipped. 6 appears to be bone index, 7 appears to be bone weight. Normals appear to not be stored in the mesh at all then?

Edit 2: I now understand what @DKDave meant. There are two exactly identical copies of the (outside) ship mesh (0xE12079E3F967650C:6, 0xA1AD36AF446FCB7D:6), however one is offset by 8 bytes. The isn't a noticable difference in datatype, so there's something in the mesh entry, and adjusting the offset by exactly 8 fixes it. That single field appears to introduce 2 floats, or 4 halfs, or 8 bytes.

Edit 3: Well, this is certainly frustrating. I've found several of these problems (all with exact clones of the same model), and I believe it might be a problem with the BMS script. The only difference between working and not working is an 8 byte change at 0x20, switching either between being a copy of 0x8, or being all zeroes. Guess someone needs to figure out a fix to the BMS script.

Edited February 26 by Xaymar

Helldiver · February 26

2 hours ago, Myrkur said:

you happen to also know where the regular dog (gun variant) is, and the hellpods are?

a14f1a98c80575af dog

for hellpod you can check

7abbdd092bcd1bd9
f6fb08cc02d24255
fc5b6bff0db90aab

Edited February 26 by Helldiver

Xaymar · February 26

Hmm, it seems that the BMS script is sometimes outputting invalid .mesh files, which have what appears to be 4 half/uint16_t's at the start. I figured out a lot about the mesh information structure while debugging it, but I found nothing that even remotely pointed at an offset of exactly 8 bytes.

Edit: I'm now uploading the 010 Editor templates to the same repository as Hellextractor itself. You'll find the latest version of my templates here, and it may end up updated multiple times a day.

Edited February 26 by Xaymar

Myrkur · February 26

1 hour ago, Helldiver said:

a14f1a98c80575af dog

for hellpod you can check

7abbdd092bcd1bd9

f6fb08cc02d24255

fc5b6bff0db90aab

thanks a bunch

Myrkur · February 26

On 2/23/2024 at 8:27 AM, h3x3r said:

@Xaymar

This file: 0d48fc8d98505493 has 32bit indices

Also discovered that files without *.gpu_resources are mostly audio/video containers. So there's no point of unpacking it since there is no model/texture.

how is audio extracted? hoping its not audio banks got ptsd from them.

Helldiver · February 26

1 minute ago, Myrkur said:

how is audio extracted? hoping its not audio banks got ptsd from them.

It's wwise.

Sign In

Helldivers 2 Model Extraction Help!

Recommended Posts

Link to comment

Share on other sites

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Popular Days

Popular Posts

Unordinal

Xaymar

CosmicDreams

Posted Images

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Create an account or sign in to comment

Create an account

Sign in

Announcements

ResHax.com: Empowering Curious Minds in the World of Reverse Engineering

Useful links