Outfit7 starlite engine (pre-2023) 3d models (My Talking Tom 2/My Talking Angela 2 etc)

[yarcunham]

I have managed to reverse the 3d files fairly well. I can get the vertices, normals, faces and (multiple) uvs out of the files, but there is still some mystery data in there.

The files are in little endian order. This is a screenshot of the hex dump of a very minimal 3d model. It's a quad: 4 vertices, 2 faces, 4 uv coordinates. No normals, no skinning data.

The first 2 things I've surrounded here are some kind of asset meta data, header and footer. The only field whose meaning I know is the first 4 bytes, which is the size of the payload. (320 in the screenshot, highlighted with a peach background)

The first 120 bytes of the mesh file I have just ignored, that has worked so far.

In this surrounded area, the first field with the light blue background, is the size of the vertex data. In this case it is 48 bytes, or 0x30. The field's offset is 136 (0x88) from the start of the file including the meta header, 120 (0x78) from the start of the payload.

The second field, with green backround is the start of the face data, expressed as an offset from the start of the payload. Its value is 304, or 0x130. If you seek from the start of the file, add the header's size, 16 (0x10) to the offset.

The 3rd field, in a darker blue, is the number of "face entries", 6 in this case. It's a number divisible by 3 because every group of 3 entries defines 1 face. 6/3 = 2. 2 faces.

The 4th field, in red, is the "vertex stride", 12 (0xC), which tells you how many space a single vertex takes in the file. The vertex data size above is evenly divisable by the vertex stride: 48 bytes/12 bytes = 4. 4 vertices.

The next 18 16-bit values starting from file offset 234, 0xEA (payload offset 218, 0xDA) tell what data is present in each vertex, 0xff means that the field is not present in the data. The fields are, starting with the blue field whose value is 0x0193:

• position data type,
• (red) normal data type
• the next 8 fields are unknown, but present in some data. I assume they are some kind of skinning data, but I haven't been able to figure them out.
• (green, value 0x0109) uv0 data type
• (brown, light blue, green) uv1, uv2, uv3 data type
• the last 5 fields have always been 0xff in the files I've seen.

The values in those fields that I've observed are, and their probable meanings. The sizes are accurate, meaning that if you add up the sizes of the vertex data present, it should add up to exactly "vertex stride" defined above.

• 0xff: not present
• 0x0021: 2D vector with float values (x/y, u/v) size: 8 bytes
• 0x0022: 3D vector with float values (x/y/z) size: 12 bytes
• 0x0023: 4D vector with float values (x/y/z/w) size: 16 bytes
• 0x0083: (unsure): 4D vector with u8 values (x/y/z/w) size: 4 bytes
• 0x0103: (unsure): 2D vector with u16 values (x/y, u/v), size: 4 bytes
• 0x0183: (unsure): 3D vector with s8 values and 1 byte of padding (x/y/z), size: 4 bytes
• 0x0191: 2D vector with S16 values (x/y, u/v), size 4 bytes.
• 0x0192: 3D vector with S16 values (x/y/z), size 6 bytes.
• 0x0193: 4D vector with S16 values (x/y/z/w), size 8 bytes.

To convert any of the "S16" vectors to float, divide their components by 32767f. With the 8-bit values, divide by 127f.

So, looking at the data table, we can see that the vertex data in this case is: positions using the 0x0193 4D-vector (the W-component is unused), and UV0 with the 0x0191 2D-vector. Total size: 8 + 4 bytes = 12, which is equal to the vertex stride defined earlier in the file.

Next section has the vertex data, I have separately highlighted the firt vertex:

The purple, brown and pink fields are the x, y and z positions of the first vertex. The following 0 is unused/padding. The position of the first vertex is (after dividing all the values by 32768) is 0.34864, 0, 0.87246. After the position (and padding) come the U (blue) and V (orange) coordinates of the vertex: 0,0. The other 3 vertices follow the same pattern.

The last section in the payload of this file is the face defitions. Nothing surprising here: 3 index values that refer to the vertices defined earlier, 2 times because there are 2 faces: 2->1->0 and 3->0->2.

The last part of the whole file is the footer of the meta file container. I don't know what any of these values mean, but so far it hasn't been a problem.

[bobo]

"Yes~ Although you have described this file in great detail, I think perhaps you should send a sample file so that people can observe it better."

Edited January 7 by bobo

[yarcunham]

Two flaws I have noticed in my process: all the UV maps are flipped on the Y-axis and all the 3D models are flipped on the X-axis.

I did notice the UV map flipping almost immediately, because the textures didn't line up, but I only noticed the 3D-model X-axis issue because I looked at some screenshots from the game and noticed that the environments are flipped.

So: I suppose the Starlite engine uses left-handed coordinates. Positive X-axis goes "left" in 3D space. I'm not sure if it's Z-up or Y-up, because I'm outputting model files that I am then importing into Blender to check my work and that conversion process can add its own changes. I suppose I could try doing the importing directly in Blender using python.

Anyway, left-handed coordinates also explain the UV map Y-axis issue, where 0,0 would be the top-left corner and X increases to the right and Y increases down

[yarcunham]

# see improved script in post below

This script makes a very rudimentary import from the file format to blender. It reads normals and uvs, but does not set them to the mesh. It also reads and stores all the unknown data. Maybe it can be saved as attributes on the vertices and inspected that way.
the "yourFileHere.dat" is one of the files you have extracted from the assets.bundle file.

assets.bundle has a simple format: u32 decompressed_size, u32 compressed_size, 64bit hash value, [compressed_size bytes of potentially zstd compressed data]. if the uncompressed and compressed sizes match, then that chunk is uncompressed. if the uncompressed size is 65536 bytes, then concatenate this chunk with the following chunks until you find one that is not 65536 bytes.

Edited January 8 by yarcunham
removed outdated script

[yarcunham]

# updated again, below

This is an improved version of the import script. It fixes the up-axis and handedness to match Blender's conventions. It also applies custom normals and uv maps. This is as far as I have gotten in figuring out the data format. I suppose I could try adding the extra vertex data as attributes to the vertices in Blender to try to figure out what's the deal with it.

Edited January 8 by yarcunham
updated the script again

[yarcunham]

(code removed, see page 2 for the latest version)

Okay, this is as far as I have gotten. Now the unknown properties are stored as vertex attributes and you can look at them in geometry nodes spreadsheet and you can use them in the shader with the attribute node.

The data UBYTE4 data type would probably be better stored as a byte color value rather than a quaternion, but whatever

Edited February 10 by yarcunham
removed outdated script

[yarcunham]

(skeleton import script removed, find an updated version below)
This skeleton import script works most of the way. It definitely gets the positions right, except the skeletons are oriented along the Y-axis and not the Z-axis and the left and right side of the skeleton is flipped. Same issue as with the meshes, but I'm less confident about flipping the skeleton since rotations get really broken when they break.

There is the unkown data that I skip while parsing the file and its length is 16 * (number_of_bones - 1), so that could actually be the orientation of the all the bones except for the root bone, maybe as a 4-component short or half precision float

Edited January 25 by yarcunham
removed outdated script

[bobo]

Yes, it's very valuable for learning. I'll take a good look at this article. In fact, I've been paying attention to your research for a long time.

[yarcunham]

(animation import script deleted, find an updated version below)

This script is meant to import animations to an armature, but because all the bone orientations are all kinds of broken, this might not work at all. It "works" in the sense that it reads the animation file and assigns keyframes to the bones in the selected armature using the values it reads from the animation file. But the animation itself is just all kinds of busted.

Potential causes are: the animation is read correctly, but because the resting orientation of the bones is wrong, the animations break more and more the further down the bone chain you get. The animation isn't even read correctly and then the bone orientations multiply already wrong rotations event more.

And if the animation isn't read correctly, here are a couple of possible causes: The decompression of the animation from the ozz-animation file isn't right (I tried porting the c++ code to pyhton, but I could have misunderstood it), the handedness and up-axis issue, again. And lastly: ozz-animation orders its quaterninons in XYZW and blender in WXYZ, but I already reorder the W component, so maybe that isn't it.

Edited January 25 by yarcunham
removed outdated script

[yarcunham]

I could attach some example files for the model, the skeleton and the animation, but I don't want to break the rules

[yarcunham]

Oh yeah, one thing I have noticed while trying to figure out what the unknown vertex data means is that one of the data types only seems to have values that are divisible by 3. So 0, 3, 6, 9, 12, 15 and so on. It makes me feel like it's some kind of an offset to an array where the values are 3 elements long, but I don't know, I'm flailing.

[yarcunham]

The mesh files, skeleton and animations are in the attached 7z file. I have not been able to progress much in the last week or so. I have identified a couple of new pieces of data in the mesh files:

at offset 0x50 there is a u32 that points to a data block that contains some kind of 4x4 (presumably transformation) matrices.

Offset 0x58 holds a u32 that tells the number of transformation matrices.
Here is a screenshot of the matrices in the "corneas" mesh file:

As you can see, the diagonal values are float 1 (00 00 80 3f). I don't know the relevance of this data unfortunately.

Offset 0x60 hods a u32 that tells the offset of some data block after the matrix. There is a u32 value at 0x68 that probably tells the number of data in the block, but I haven't been able to reliably figure out the size of a single piece of data.

Offset 0x70 holds a u32 tells the offset of some  other data block after the first. Again there is a u32 value at 0x78 that tells the number of data in the block.

Also the skeleton files have some unknown data in them before the ozz skeleton data.

I have to assume that some combination of the unknown vertex data + the matrices + the 2 "after matrix data" + maybe the extra data in the skeleton file allows for binding the mesh to the skeleton.

I have also not been able to animate the skeleton properly. I'm pretty sure that starlite engine uses left-handed Y-up coordinates. Whatever the case, I have not been able to convert the transformations in the animation files into a form that produces anything except a glichy mess

MTA2-angela.7z

[yarcunham]

Lol, I guess "Left-handed Y-up confirmed" by this blog post: https://outfit7.com/blog/tech/building-the-ultimate-mobile-game-engine-starlite

Relevant screenshot from the editor:

Note the axis gadgets in the center and corner of the main editor screen. Y is up, Z is toward the bottom left, X is toward the top left.

[yarcunham]

I found one major problem in the animation import script: blender does not preserve the indices of bones when you change their parents, so I have been assigning keyframes to entirely wrong bones the whole time. The first 6 bones seem like they have the same index, then it gets wildly out of sync. This is some of the output of a debugging script, where the first number is the original index, the second number is the index blender assigns to the bone and the string is the bone's name:
 

(0, 0, 'root'),
(1, 1, 'C_main_root__SET'),
(2, 2, 'C_skin_joints__SET'),
(3, 3, 'c_root_uJnt'),
(4, 4, 'c_addon_uJnt'),
(5, 5, 'c_data_uJnt'),
(6, 6, 'c_offset_uJnt'),
(7, 242, 'l_armAddon_uJnt'),
(8, 243, 'c_camera_uJnt'),
(9, 244, 'r_armAddon_uJnt'),
(10, 7, 'c_hip_00_uJnt'),
(11, 8, 'rb_light_top_uJnt'),
(12, 12, 'cf_light_top_uJnt'),
(13, 16, 'lf_light_top_uJnt'),
(14, 20, 'cb_light_top_uJnt'),
(15, 24, 'cb_tight_top_uJnt'),
(16, 28, 'lb_tight_top_uJnt'),
(17, 32, 'ls_medium_top_uJnt'),
(18, 36, 'cf_tight_top_uJnt'),
(19, 40, 'rf_light_top_uJnt'),
(20, 44, 'cb_medium_top_uJnt'),
(21, 48, 'lb_light_top_uJnt'),
(22, 52, 'r_femurRibbon_01_uJnt'),
(23, 62, 'lf_medium_top_uJnt'),
(24, 66, 'c_tail00_uJnt'),

 

[yarcunham]

I'm still kind of stuck, but I'm going to post the current best working version of my skeleton and animation import scripts. The old animation import script was just completely broken, the rotations were not even close to being correct. There were double conversions, wrong element order, reading data from wrong offsets. All kinds of garbage. This current version isn't as broken. If I convert a simple animation using the ozz animation tools and then import it with the script, it... mostly works.

As for the skeleton import, I'm fairly sure that I get the positions correct. But the bone orientations are wrong and I think they might be the data that the script skips over. The data size is (num_bones - 1) * 8 bytes. I interpret that as having 8 bytes of data for all bones except the root bone. It could be orientations or it could be some kind of a bone id for mesh binding.

Anyway, here is the current best version of the skeleton import script:

And here is the animation import script:
((go to post https://reshax.com/topic/1418-outfit7-starlite-engine-pre-2023-3d-models-my-talking-tom-2my-talking-angela-2-etc/page/2/#findComment-8332 to dowload the latest scripts)

Edited February 28 by yarcunham
removed outdated scripts

[yarcunham]

Tiny bit of progress: looks like at least some animation keyframes are in model space. I was trying to figure out the bone orientation issue and I noticed that if I oriented some bone chains so that for example their X-axis pointed toward their parents, then that bone chain would just collapse into the first joint of the chain when an animation was applied to the bones. Those collapsed joints would have a translation keyframe value of for example (0.25, 0, 0).

So the joint position is not meant to be added on top of the rest pose, which feels weird to me. If I set the positions of the joints to (0,0,0), then the proportions of the skeleton during animation playback seem sensible. Still struggling with the resting orientations of the bones.

[yarcunham]

On 1/19/2025 at 10:11 PM, yarcunham said:

Offset 0x60 hods a u32 that tells the offset of some data block after the matrix. There is a u32 value at 0x68 that probably tells the number of data in the block, but I haven't been able to reliably figure out the size of a single piece of data.

Offset 0x70 holds a u32 tells the offset of some  other data block after the first. Again there is a u32 value at 0x78 that tells the number of data in the block.

Also the skeleton files have some unknown data in them before the ozz skeleton data.

I have to assume that some combination of the unknown vertex data + the matrices + the 2 "after matrix data" + maybe the extra data in the skeleton file allows for binding the mesh to the skeleton.

I'm pretty sure that the data in the skeleton files assigns an 64-bit identifier to each joint in the skeleton file (except for the root joint). And the "data after the matrices" in the mesh file then references the same 64-bit identifiers.

[scratchcat579]

i was also working on an importer but it's unity because i don't want to learn python

(i suck at c#, i know.)

it imports vertex information: pos, normals, uv0 and 1, tangents, skin indices and weights, vertex color.

it imports faces ushort.

it imports bones but the bindposes i havent figured out what they mean, but i make it generate bones.

sometimes the skinning is broken when imported.

it also imports blendshapes with names.

i attached the source code of the importer and images of an imported model.

edit: at line 341, replace the indices code with the following:

indices.x = (indices.x / 3 % indicesCount);
indices.y = (indices.y / 3 % indicesCount);
indices.z = (indices.z / 3 % indicesCount);
indices.w = (indices.w / 3 % indicesCount);

starlite mesh to unity 3d mesh.zip

Edited February 3 by scratchcat579

[scratchcat579]

On 1/7/2025 at 10:03 PM, yarcunham said:

# see improved script in post below

This script makes a very rudimentary import from the file format to blender. It reads normals and uvs, but does not set them to the mesh. It also reads and stores all the unknown data. Maybe it can be saved as attributes on the vertices and inspected that way.
the "yourFileHere.dat" is one of the files you have extracted from the assets.bundle file.

assets.bundle has a simple format: u32 decompressed_size, u32 compressed_size, 64bit hash value, [compressed_size bytes of potentially zstd compressed data]. if the uncompressed and compressed sizes match, then that chunk is uncompressed. if the uncompressed size is 65536 bytes, then concatenate this chunk with the following chunks until you find one that is not 65536 bytes.

how did u get the file names for the angela 2 assets? (what is the code you used to extract the assets, it is Moonlite or your own code)

[yarcunham]

On 2/3/2025 at 2:50 AM, scratchcat579 said:

how did u get the file names for the angela 2 assets? (what is the code you used to extract the assets, it is Moonlite or your own code)

I did the dumbest thing: I mostly did it manually.

That said: to get started, I unpacked the root.bundle file with the same code I used to unpack root.assets. Then I used another utility to dump all the "aligned strings" inside the unpacked file. The strings utility works too, but all the file names in root.bundle are aligned to 16 bytes, so I used that to eliminate some false positives

The strings are mostly file names and they're mostly in the same order as the files from root.assets. I have not really tried to figure out the exact format since the dumb method works so well. By "mostly" I mean that If I go file by file in numerical order, at some points I get off-by-one errors when trying to figure out a file's name.

Anyway, since I have files that I have extracted from root.assets and I have a list of file names, I just browsed through all the png and jpeg textures I had extracted until I found something I recognized, like a butterfly. Then I searched for butterfly in the file list and got a match.

"ok, so '01783_97a3f38252f65813.png' is probably 'Bundles/Global/Wardrobe/TextureData/Stamps/wardrobe-stamp-butterfly', oh and the previous file '01782_f9c59db35d19eaa8.png' lines up with 'Bundles/Global/Wardrobe/TextureData/Stamps/wardrobe-stamp-heartNeon'"

Then I started working forward and backward from those. Like I said, the dumbest thing.

[yarcunham]

On 2/3/2025 at 1:49 AM, scratchcat579 said:

i was also working on an importer but it's unity because i don't want to learn python

(i suck at c#, i know.)

it imports vertex information: pos, normals, uv0 and 1, tangents, skin indices and weights, vertex color.

it imports faces ushort.

it imports bones but the bindposes i havent figured out what they mean, but i make it generate bones.

sometimes the skinning is broken when imported.

it also imports blendshapes with names.

i attached the source code of the importer and images of an imported model

Awesome, btw.

I'm using python because that's what Blender uses and I can use it as my 3d visualizer. And if I can get it working then anything imported into Blender can be exported to anything else, basically.

I haven't looked at your code yet, but I'm really interested in the weighting and the bone import, because that's what I have been stuck on for so long.

I get the bone positions correct, but I can't get animations working well. With some simple skeletons I can get animations to work pretty well, but the bone alignments are wrong.

I'm not sure if my initial bone alignments are wrong or if I read the animation data into the wrong bones.

By alignment being wrong, this is what I mean:

If an animation keyframe says "rotate -90 degrees around the (local) X axis"

Then If your bone alignment is this:

Your result is this:


But if your bone alignment is this instead (rotated 90 degrees along the bone's Y axis, note the Z and X axes pointing in other directions):

Your result would be this:

Multiply that error by every bone in a bone chain and your idle animation truns into John Carpenter's The Thing

Edited February 5 by yarcunham
clarification

[yarcunham]

Now that I took a look at the C# code posted, I renamed some of my enums and analyzed all the asset files and here are all the data types encountered for the various vertex data fields. I also added values from the ChannelType enum on the line. I didn't correlate my field names "unkn0...11" to the field names defined in the C# code yet.

seen vertex data types: 
{
	'position': {
		<VertexData.FLOAT3: 34>: 824, (FloatVector3)
		<VertexData.SHORT3: 402>: 1178, (SignedShortVector3)
		<VertexData.SHORT4: 403>: 7, (SignedShortVector4)
		<VertexData.VERTEX_COLOR: 131>: 2 (VertexColor)
	},
	'normal': {
		<VertexData.SHORT4: 403>: 629, (SignedShortVector4)
		<VertexData.SHORT3: 402>: 1178, (SignedShortVector3)
		<VertexData.FLOAT3: 34>: 192, (FloatVector3)
		<VertexData.SBYTE3_PAD: 387>: 2 (Not defined)
	},
	'unkn0': {
		<VertexData.SHORT4: 403>: 8, (SignedShortVector4)
		<VertexData.FLOAT4: 35>: 1 (FloatVector4)
	},
	'unkn1': {},
	'unkn2': {<VertexData.VERTEX_COLOR: 131>: 1078}, (VertexColor)
	'unkn3': {<VertexData.VERTEX_COLOR: 131>: 15}, (VertexColor)
	'unkn4': {<VertexData.VERTEX_COLOR: 131>: 5}, (VertexColor)
	'unkn5': {<VertexData.VERTEX_COLOR: 131>: 3}, (VertexColor)
	'unkn6': {<VertexData.SKIN_INDICES: 259>: 397}, (SkinIndices)
	'unkn7': {<VertexData.FLOAT4: 35>: 240},
	'uv0': {
		<VertexData.SHORT2: 401>: 1051, (SignedShortVector2)
		<VertexData.FLOAT2: 33>: 207 (FloatVector2)
	},
	'uv1': {
		<VertexData.SHORT2: 401>: 378, (SignedShortVector2)
		<VertexData.FLOAT2: 33>: 13 (FloatVector2)
	},
	'uv2': {<VertexData.SHORT2: 401>: 6}, (SignedShortVector2)
	'uv3': {<VertexData.SHORT2: 401>: 5}, (SignedShortVector2)
	'unkn8': {},
	'unkn9': {},
	'unkn10': {},
	'unkn11': {}
}

A couple of notes: there are 2 meshes that define their vertex positions using the data type we have called vertex color.
2 meshes define their normals in what I've called SBYTE3_PAD format where the vector is defined with 3 bytes and then there is an empty padding byte. This enum value (0x0183) is not defined in the C# code.
Also, looks like the fields unkn1, unkn8... unkn11 are unused

[yarcunham]

On 2/3/2025 at 1:49 AM, scratchcat579 said:

it imports bones but the bindposes i havent figured out what they mean, but i make it generate bones.

 

The bind pose matrices are either the transformation matrix of the bone the vertices are supposed to be bound to, or more likely they're the inverse matrix because that is static and needed for animation: You multiply the vertex position with the inverse transformation matrix of the bone, then you multiply it with the transformation matrix of the bone at the current frame.

The mesh file also has some unknown data after the matrices, which I have just called "after matrix", and after that there are 64-bit joint ids, which correspond to the joint ids defined in the skeleton files (when you can find by searching for "ozz-skeleton" in the dumped asset files)

Here is an annotated picture of where the offsets and numbers are defined in the header:

edit:

For example, If I look at the entry number 17 in the joint id list, it's BE4303DC94F197F0, If I then search for it in the skeleton file, it's entry number 74. Because the root bone doesn't get its own entry, I look at entry number 75 in the skeleton joint name list and that is r_tibiaRibbonTweak_01_uJnt. So that is how you would bind any vertex that references bone BE4303DC94F197F0.

Edited February 6 by yarcunham
clarification

[scratchcat579]

i exported fbx files from unity and i tried using the imported skeleton generated from the bindposes.

the new source code is here: https://github.com/nj-josh-man/MeshImporterForUnity/

 

 

Edited February 7 by scratchcat579

[scratchcat579]

the 20 float bytes in the header are actually bounds.

my importer script uses them.