[Xbox] Conker Live and Reloaded models

[birdytsc]

Looking to see if anyone is interested in this, 3D models/geometry isn't exactly my strongest skillset (which I'm sure you will soon notice).

I was originally inspired by an old thread on Xentax for other Rareware games (primarily Grabbed by the Ghoulies) however Live and Reloaded seemed to get skipped over since no one could figure out the compression the game was using.

Fast forward a few years and i learned that Xemu had a debugging feature with support for IDA which is a tool I'm fairly familiar with, using this i was able to extract the x86 assembly instructions the game uses to decompress the game assets and write a wrapper for it allowing me to unpack the files to a state similar to what's used in other Rareware titles.

You can check out that tool here: https://github.com/birdytsc/clr_unpack

I'm currently in the process of reversing the games model format but my limited experience with 3d models/formats is hindering my progress.

---

While some of the data is pretty straightforward to obtain some are packed away in pushbuffer instructions, one example is the tri's/faces data:

This makes it a little more annoying to tell what set of vertexes they belong to but they are (so far) stored in order that you will find any vertex data

one problem i am having is im not sure whether im extracting the data properly,

example:

the first part for conkers main model is broken into 3 sets of triangles all sharing a single set of vertexes, uv's etc.

the model only looks correct if i separate the 3 sets of tri's into there own mesh, keeping them together creates artifacts:

---

Another issue im having is Texture alignment, the game stores the textures as raw DXT data (usually DXT1 or DXT5)

even in simpler models that only consist of a single set of triangles im still having alignment issues, for example:

Perhaps this is a sign that there is indeed an issue with how im extracting the Tri's?

---

If anyone wants to play around with this stuff ive included some files in the zip:

• aid_bfdmodel_characters_conker.py -  a script for ModelResearcherPro that will load the verts/tris for the main conker model
• retail_aid_zpackage_general_singleplayer.rbm.listfile.csv - list of file names (if they havnt been stripped from the asset packer) and offsets
• retail_aid_zpackage_general_singleplayer.rbm.unpacked - just the decompressed version of the file found in the retail game
• retail_aid_zpackage_general_singleplayer.rbm.unpacked.mapped - same file as above but after its had a bunch of pointers mapped, useful for diffing against the original to find pointers.
• retail_aid_zpackage_general_singleplayer.rbm.unpacked.mapped.hexpat - imhex pattern file for the previous file.

 

 

conker_stuff.zip

[Shadowth117]

So just looking at what you've posted and written, can't look in depth at the moment...
-Model separation might be okay. Sometimes you do get some funny separation going on and it's not always readily sensible

-UVs might be correct, worth checking it against the actual game. But if by comparison to the game they seem wrong, I'd say to look at messing with how they're being read. It looks a little based on that last image of the turret texture overlay like UV scale might be slightly off, causing things to stretch a tad out of bounds. If the UVs aren't just floating point values I'd definitely play with that with a screenshot from ingame onhand.

Cool stuff though, wishing you luck.

[birdytsc]

10 hours ago, Shadowth117 said:

-Model separation might be okay. Sometimes you do get some funny separation going on and it's not always readily sensible

Yeah I'm still not sure why they're packed the way they are exactly, i've found some source code online for some xbox titles but they dont seem to setup there pushbuffers the same way.

example

https://github.com/zachlatta/jedi-academy/blob/master/code/win32/win_qgl_dx8.cpp#L859

however i did stumble upon this interview where the devs talked about xbox optimizations:

https://www.gamedeveloper.com/production/porting-a-ps2centric-game-to-the-xbox-a-case-study-of-state-of-emergency

of particular interest was this quote:

Quote

Pushbuffers. Pushbuffers are the native rendering format of the Xbox GPU. When DirectX is used on the Xbox, the rendering instructions are converted into pushbuffers for execution on the GPU. A small subset of commands within the main render loop: draw primitive, z-writes, set vertexbuffer, select vertexshader, set pixelshader, etc, were reverse engineered to determine the resulting pushbuffer code. This was then used to develop an offline converter application that constructs a large pushbuffer for the whole level. Each object in the pushbuffer is preceded by a null operation (NOP), which is conditionally set as a NOP, if the following object is visible, and a jump, if the following object is not visible. The concept is similar to "execute buffers" rendering techniques in DirectX 3.0. This pushbuffer technique resulted in a reduction in the CPU set up time required for the rendering, essentially halving the frame time.

it caught my attention because in a lot of scenegraph files theres a reference to something they call a "ndStaticPushbuffer" which usually contains a pointer(as well as the size of) to the start of a large array of pushbuffer instructions and the first thing in the array is a nv2a "NOP" instruction and it seems likely that every pointer in the scengraph file related to that section of data i.e where the tri's are location are relative to this starting position.

for example, so far the first pushbuffer instruction always seems to be

#define NV2A_NOINCREMENT_FLAG = 0x40000000
#define NV097_NO_OPERATION = 0x100;
D3DPUSH_ENCODE(NV2A_NOINCREMENT_FLAG | NV097_NO_OPERATION, 576);

perhaps they use this as some kind of reserved space to write additional pushbuffer commands later since this is initially loaded into memory as-is?

if you follow the scenegraph chain youll find something like:

ndVertexBuffer2(which has the verts/normals/uvs)

  -> ndShaderParam2

    -> ndRigidSkinIdx

        -> ndVisibility (pointer to an additional noop?)

        -> ndVisibility (sibling to the first which contains a relative offset to the Tri faces.)

if you look at my OP tho it doesnt actually point to the first NV097_SET_BEGIN_END(0x17FC) but rather the 0x0100(NV097_NO_OPERATION) right before it, so this seems like this is infact the same technique the developers were talking about in that interview as a technique to boost performance?

--

I'm also still unsure why they are packed the way they are however, for example, there appears to be a 2047 limit on elements for a single pushbuffer instruction according to comments here:

https://github.com/thug1src/thug/blob/d8eb7147663d28c5cff3249a6df7d98e692741cb/Code/Gfx/XBox/NX/screenfx.cpp#L142

probably due to the last 2 bits being used for flags leaving only 14bits to encode the size?

--

but in most cases they seem to just be loaded arbitrarily.

for example, one part of the model might have 1893 faces but they are not loaded in multiples of 2047 but instead loaded as

 

NV097_ARRAY_ELEMENT16 -> 8

NV097_ARRAY_ELEMENT16 -> 862

NV097_ARRAY_ELEMENT16 -> 1022

NV097_ARRAY_ELEMENT32 -> 1 (why 32bit?)

 

 

im completely baffled as to why they do it this way when it seems they could do it in fewer pushbuffer operations? and why is the last index often 32bit? if its read as 32bit the index is never high enough to need more than 16bits, and if its read as 2x 16 bits the last index is always 0x0000, would there be any significance of this?

-------------

side note, i still cant quite figure out the UV mapping, it seems like the texture coords maybe reference only portion of the whole texture?

the full texture is larger however, at 256x256

there also appears to be multiple lods in the DXT texture data:

Edited 15 hours ago by birdytsc