reversing 3D model from the executable/d3d calls

[V12-POWER]

hi all, question to some experts. I was wondering what would be the process of reversing a 3D mesh, by debugging the exe, for example, I already know some properties of the 3D mesh format im trying to reverse (this is basic stuff like, vertexstride, vertex buffer size and offset, indices size and offset, and the subsets vertex offset, index offset, etc) that I can see from the 3d file itself.

But the data in the vertex buffer is weird and being the average vertex stride of 48 bytes, there is a lot of data aside from coordinates, which are not even float values. theyre int16. 

so instead of guessing around and just because, im debugging the game, got to the point where the vertex buffer is loaded to a separate memory space by the d3d11.dll, but then after that im not able to "grab" the moment its actually reading the data from the buffer. 

What am I missing here? why isn't the program stopping at a breakpoint in the d3d data? even if it was passed to a shader, shouldnt I still be able to grab it in that moment? 

[shak-otay]

11 hours ago, V12-POWER said:

hi all, ...

got to the point where the vertex buffer is loaded to a separate memory space by the d3d11.dll, but then after that im not able to "grab" the moment its actually reading the data from the buffer. 

Hi. Great!

Quote

What am I missing here? why isn't the program stopping at a breakpoint in the d3d data? even if it was passed to a shader, shouldnt I still be able to grab it in that moment? 

Where is your breakpoint? In the separate memory space?

What kind of breakpoint? "break on access"?

[V12-POWER]

9 hours ago, shak-otay said:

Hi. Great!

Where is your breakpoint? In the separate memory space?

What kind of breakpoint? "break on access"?

 

hey man. the game first loads the file into a memory space and reads some information, like the mesh subset names (theyre in plain ascii), vertex offsets, index offset, vertex count, vertex strido and some more data, maybe bones and primitives. 

but after that, it will pass parameters to the d3d11.dll, those are vertex buffer size and index buffer size. 

i put a breakpoint on access (hardware bp) on the initial memory space, on the vertex buffer data. with this I can get to the point where the d3d11.dll copies it, from memory space 1 to a different place. When it copies, i place another bp on access on the freshly copied data, which is the 1st byte of the vertex buffer, then hit run and the game doesnt seem to be reading from it? it just loads and plays normally. I would expect to hit the breakpoint at some point but it just doesnt. (I tried putting breakpoints at different points of the vertex buffer but it still doesnt hit) 

im using x32dbg btw. is this is because this data goes to the GPU and the debugger cant detect it? im a noob when it comes to rendering, so some parts of the process I simply dont know. thanks 

 

[shak-otay]

9 hours ago, V12-POWER said:

im using x32dbg btw. is this is because this data goes to the GPU and the debugger cant detect it?

Hi,

that's hard to tell from "the far". I'd suggest to try it out with CheatEngine.

Don't laugh - it comes with a good debugger. "Break and trace" for your last working breakpoint might help.

[V12-POWER]

On 9/12/2024 at 6:57 AM, shak-otay said:

Hi,

that's hard to tell from "the far". I'd suggest to try it out with CheatEngine.

Don't laugh - it comes with a good debugger. "Break and trace" for your last working breakpoint might help.

 

ok I will try using CE this weekend if I have time. thanks 

[V12-POWER]

On 9/12/2024 at 6:57 AM, shak-otay said:

Hi,

that's hard to tell from "the far". I'd suggest to try it out with CheatEngine.

Don't laugh - it comes with a good debugger. "Break and trace" for your last working breakpoint might help.

okay so I could not get it that to work with CE. then i tried hex2obj but im not getting proper results. the file attached has the vertex buffer at offset 0x284, a stride of 48 bytes, index buffer at 0x1FB434 and the following structure:

first 6 bytes -> int16 

int16 half float (1 or -1) 

next 6 bytes is a vector 

int16 half float (1 or -1) 

next 6 bytes is a vector 

int16 half float (1 or -1) 

then the next 8 bytes mighte UV coordinates? im not sure. there are also 4 bytes which i believe are vertex colors. 

but using this information in hex2obj outputs this: 

https://ibb.co/kXzm99S

can you check why this is happening? im kinda lost, also im not sure why the max face index does not match the total number of vertices (size/stride = 43273) and not 42840 like the tool says. edit: this is a frostbite 1-1.5 mesh. so fb2 tools are not compatible

edit2: what is the math behind converting unsigned ints to float values? im not able to find anything on the subject

 

Mi24_Mesh_lod0_data.rar

Edited September 15 by V12-POWER

[V12-POWER]

1 hour ago, V12-POWER said:

okay so I could not get it that to work with CE. then i tried hex2obj but im not getting proper results. the file attached has the vertex buffer at offset 0x284, a stride of 48 bytes, index buffer at 0x1FB434 and the following structure:

first 6 bytes -> int16 

int16 half float (1 or -1) 

next 6 bytes is a vector 

int16 half float (1 or -1) 

next 6 bytes is a vector 

int16 half float (1 or -1) 

then the next 8 bytes mighte UV coordinates? im not sure. there are also 4 bytes which i believe are vertex colors. 

but using this information in hex2obj outputs this: 

https://ibb.co/kXzm99S

can you check why this is happening? im kinda lost, also im not sure why the max face index does not match the total number of vertices (size/stride = 43273) and not 42840 like the tool says. edit: this is a frostbite 1-1.5 mesh. so fb2 tools are not compatible

edit2: what is the math behind converting unsigned ints to float values? im not able to find anything on the subject

 

Mi24_Mesh_lod0_data.rar 586.29 kB · 0 downloads

nevermind. its working now https://ibb.co/Wgc1sgZ

still. im not sure what the tool is doing when I put HF_all it works, with ShortAll it doesnt work, but the value range is very small so surely im missing something? 

(im trying to make a script to import geometry in the game, not just extracting) 

 

 

Edited September 16 by V12-POWER

[Durik256]

1 hour ago, V12-POWER said:

but using this information in hex2obj outputs this: 

use halffloat

multiple UV channels possible

 

Edited September 16 by Durik256

[V12-POWER]

1 hour ago, Durik256 said:

use halffloat

multiple UV channels possible

 

yeah I figured it out...seems I was wrong about coords being integers. 

so it seems it wont be that hard to make a script for custom meshes. but im not understanding the output of the face indices of hex2obj. like why is it adding 1 to each index

the offset 0x1FB434 starts like this 

0000 0100 0200

the output of hex2obj is 

1 2 3 

the next 6 bytes at offset 0x1FB43A are:

0300 0400 0100

the output of hex2obj is 

4 5 2 

......

0x22426A:

060009000700

hex2obj: 

7 10 8 

 

what is actually going on? shouldn't the output be

0x1FB434: 0 1 2 

0x1FB43A: 3 4 1? 

 

 

 

Edited September 16 by V12-POWER

[V12-POWER]

delete

Edited September 16 by V12-POWER

[shak-otay]

9 hours ago, V12-POWER said:

so it seems it wont be that hard to make a script for custom meshes. but im not understanding the output of the face indices of hex2obj. like why is it adding 1 to each index

the offset 0x1FB434 starts like this 

0000 0100 0200

the output of hex2obj is 

1 2 3 

Well, that's simply a matter of the wavefront obj format. It expects the indices to start from 1 (not from 0 like it is in the hex data).

So any converter from some3Dformat to obj has to add 1 to the face indices.

 

And for CheatEngine: after you've attached CE to a game's exe it's a litttle bit tricky to open the debugger.

You may search for a string (like dll or MZ), then follow 1), 2) (alternatively do a), b))

attach CE debugger

edit: forgot to mention: for some exe files it's required to click the CopyOnWrite checkbox once (minus appears) or no strings will be found

 

Edited September 16 by shak-otay

[h3x3r]

Can you post more samples for compare. Struct is quite simple.

EDiT: Simple except that part that face index must be further divided on parts. Because i see that model has overlaying uv sets. But not sure what controlling that.

I thought it could be these values...

 

Edited September 16 by h3x3r

[V12-POWER]

5 hours ago, h3x3r said:

Can you post more samples for compare. Struct is quite simple.

EDiT: Simple except that part that face index must be further divided on parts. Because i see that model has overlaying uv sets. But not sure what controlling that.

I thought it could be these values...

 

 

Submesh: 

Name (text) 

int32 0x0 - Faces/polygons count 

int32 0x4 - Vertex count 

int32 0x8 - Index offset

int32 0xC - vertex offset 

byte  0x10 - vertexStride
byte  0x11 - primitiveType
byte  0x12 - unknown
byte  0x13 - Array elements (what is this, I dont know)
int16 0x14 - Array elements (what is this, I dont know)

 

also, not that important but nice to know: 

Vertex layout has variations across game files. I will see if theres a flag or something in the meshdata files to determine. for example: 

Mi_24_Mesh_lod0 vertex layout 48 bytes

Vertex coordinates : DXGI_FORMAT_R16G16B16A16_FLOAT (A16 is either 1 or -1) 

Vector 1 : DXGI_FORMAT_R16G16B16A16_FLOAT (A16 is either 1 or -1) 

Vector 2: DXGI_FORMAT_R16G16B16A16_FLOAT (A16 is either 1 or -1) 

UV 1? : DXGI_FORMAT_R16G16_FLOAT

UV 2?: DXGI_FORMAT_R16G16_FLOAT

unknown: DXGI_FORMAT_R16G16B16A16_FLOAT

colors?: DXGI_FORMAT_R8G8B8A8_UINT

unknown : 4 bytes 

-------------------------------------

but now the file attached, AH64_Mesh_lod0: 

Vertex coordinates : DXGI_FORMAT_R16G16B16A16_FLOAT (A16 is either 1 or -1) 

Vector 1 : DXGI_FORMAT_R16G16B16A16_FLOAT (A16 is either 1 or -1) 

Vector 2: DXGI_FORMAT_R16G16B16A16_FLOAT (A16 is either 1 or -1) 

UV 1? : DXGI_FORMAT_R16G16_FLOAT

UV 2?: DXGI_FORMAT_R16G16_FLOAT

unknown 12 BYTES

colors?: DXGI_FORMAT_R8G8B8A8_UINT

--------------------------------------------------

RU_at_RPG7_Mesh_lod0 (48 byte vertex stride) 

Vertex coordinates : DXGI_FORMAT_R16G16B16A16_FLOAT (A16 is either 1 or -1) 

colors?: DXGI_FORMAT_R8G8B8A8_UINT

unknown 4 bytes

Vector 1 : DXGI_FORMAT_R16G16B16A16_FLOAT (A16 is either 1 or -1) 

Vector 2: DXGI_FORMAT_R16G16B16A16_FLOAT (A16 is either 1 or -1) 

UV 1? : DXGI_FORMAT_R16G16_FLOAT

UV 2?: DXGI_FORMAT_R16G16_FLOAT

 

im saying "vector" cause the sum of each component (x,y,z) squared is equal to 1. but not sure if theyre really used as such 

there are also meshes with 64 bytes stride for example some buildings, some other objects with 40, some meshes have submeshes with different vertex stride too. so theres like a shitload of variations.

but im not gonna focus on all these variations for now...I want to get one format right to begin making an in game import script. then if it works expand on the rest

 

meshes2.rar

[h3x3r]

I believe it that first 4 bytes in file is endian flag. Coz AH64_Mesh_lod0_data has big endian order.

EDiT: What's the name of the game? Battlefield 2?

Edited September 17 by h3x3r

[V12-POWER]

2 hours ago, h3x3r said:

I believe it that first 4 bytes in file is endian flag. Coz AH64_Mesh_lod0_data has big endian order.

 

sorry I must have sent an xbox360 file. I dont think the game uses big endian byte order. 

nevermind the Vertex Buffer layout, I found out where they are declared, sort of. 

 

What would a very basic import script need? vertex positions, indices and just that? or normals, etc are required? 

[V12-POWER]

On 9/16/2024 at 8:12 AM, h3x3r said:

Can you post more samples for compare. Struct is quite simple.

EDiT: Simple except that part that face index must be further divided on parts. Because i see that model has overlaying uv sets. But not sure what controlling that.

I thought it could be these values...

 

hey. had to confirm this, yes these are the sub parts of the meshset "lod0|Rotor_Main" they are indexed in a separate file, attached below, theyre not big endian, they are just an index number. 

but im not sure how we can tell which one is which. that will require some thought. I did find out that the Vertex Buffer layout is declared on yet another file, at least to recognize UV Maps. first 6 bytes are always the coordinates, the next half float is the direction the vertex is facing. 

this mesh "lod0|Rotor_Main" should have 5 uv maps, one unused I believe, the formats are very wacky, with bits spread all around. but we will figure it out. this is Battlefield Bad Company 2 

mi24.rar

[h3x3r]

Now i know how it works. It's in vertex buffer. Those 4 x uint8 are not color but shape index. I print these and min shape index was 0 > 38  which is exactly the shape index in Mi24_Mesh.compositemeshset

Edited September 18 by h3x3r

[V12-POWER]

thanks man, it wasn't that hard lol. so now it's basically all dissected. what 3d format would be appropriate for a mesh this complex? I can try and start to make an obj converter but im not sure if it supports this level of complexity? specially considering a meshdata file has many submeshes and inside submeshes there are individual shapes. 

[h3x3r]

Which file define stride type? Mi24_Mesh.dbx?

[V12-POWER]

On 9/18/2024 at 1:47 PM, h3x3r said:

Which file define stride type? Mi24_Mesh.dbx?

no its the shader database files, I am currently making a tool with a friend (slowly) to modify and create new ones to add new objects to levels. but because of this research with meshes I found the game has declarations for the buffer. 

the mesh parts get assigned an internal ID, for example Rotor_Main: 

has the Vertex Buffer layout "A51C100491A3178CAB95A15B6457EBCC" - some parts from the same .meshdata file can have different VBuffer layouts...

this layout is defined in another part of the file 

and the structure is 89 bytes for each declaration, but the relevant data is way less: 

| ID | Data Type 2 bytes | Vbuffer offset | Data type 3 bytes | Vbuffer offset | Data type 3 bytes | Vbuffer offset | ... | byte? 0x50 = Number of VBuffer offsets | byte 0x55 = VBuffer length/stride | 
 

Data type begins at the offset set afterwards 

 

EDIT: ADDED MORE DATA TYPES (there a few more STILL) 

03 01 -> Vertex Coordinates float (12 bytes) 

07 01 -> Vertex Coordinates half float (6 bytes) 

00 05 07 ->  1 or -1 Half float (2 bytes)

00 08 04 -> Vertex Normals + 4th coordinate 1 or -1 half float (8 bytes)

00 08 05 -> Vertex Tangents + 4th coordinate 1 or -1 half float (8 bytes)

00 06 00 -> UV Coordinates half float (4 bytes)

00 0C 02 -> BoneIndex/ShapeIndex (4x int8) 

00 0C 03 -> BoneWeights (4x int8)

 

the mi24 rotor_main part has the following structure (A51C100491A3178CAB95A15B6457EBCC) : 

07 01 - Vertex Coordinates half float

00 05 07 - Half float 1 or -1 

00 08 04 - Versor vector

00 08 05 - Versor vector 

00 06 00 - UV Coordinates half float

00 06 00 - UV Coordinates half float

00 06 00 - UV Coordinates half float

00 06 00 - UV Coordinates half float

00 0C 02 - ShapeIndex

00 06 00 - UV Coordinates half float

that totals 48 bytes of data per vertex just as the Vertex Stride is. I verified this to be true; the files are attached if you want to see 

there are 5 UV channels and this makes sense cause the Shapes inside Rotor_Main use all 5 textures, main rotor, tail rotor + color + normal + another gray texture, but somehow they end up crooked in hex2obj 

 

 

reshax.rar

Edited September 20 by V12-POWER

[V12-POWER]

i am able to get all the shape names, along with the parents and all the indexes. but im not familiar with the FBX SDK and i am getting tons of errors, does anyone have some knowledge with this? and that is able to help?

the tool prints out everything correctly, I need to build the scene hierarchy but i cant even get to create a scene with some cubes (from the sample programs) 

 

EDIT: it wasnt that hard...

 

Edited September 26 by V12-POWER

[V12-POWER]

 

updated now, it works on some meshes, but it has errors on some. 

 

but this one has errors: 

 

@shak-otay do you have in idea why this happens (it says overlapping polygons, fbx format) 

edit: there are improper indices? this other mesh doesnt show up properly in hex2obj either:

 

 

 

 

 

AH60_Sp_Mesh_lod0_data.rar

Edited October 3 by V12-POWER

[shak-otay]

If you start with a small block it looks proper.

The trouble seems to start at an FIcnt/vCnt of 2035/528.

XXX

edit much simpler, next FI block starts at 0x2BE634

LightsTop assumedly

part 3

(There's 15 FI blocks at elast.)

Edited October 3 by shak-otay

[V12-POWER]

15 hours ago, shak-otay said:

If you start with a small block it looks proper.

The trouble seems to start at an FIcnt/vCnt of 2035/528.

XXX

edit much simpler, next FI block starts at 0x2BE634

LightsTop assumedly

part 3

(There's 15 FI blocks at elast.)

 

ahhh i see whats wrong, need to read index buffer "individually" - thanks for ur time man, saved me days of wanking around lol

edit: it works! fuck yeah 

Edited October 3 by V12-POWER