[PS2] Gundam G-Saviour model file help

[ScornMandark]

Hi all - I've been obsessed with this little game for a very, very long time, and I'm trying to learn the file structures to try to make it work.  I've got the files broken out into the different formats from the disk, and I've pretty confidently identified the .md model files.  The mesh files themselves I'm exploring with ModelResearcherPro, and this is what I've come up with so far.  most models have multiple meshes embedded, and there's some things I have and some things I haven't figured out yet.  I'm using the attached GSG03.MD as the example model, which Iam pretty sure is a robot broken into multiple bits. (There are some models for like a map model, and it's broadly similar.)  

File Start: Little Endian
First long 26 00 00 00

Mesh Start
7x long or floats of header - this is different for each file, but very consistent in length.  First long is an address pointer to the end of the vert section, 4 bytes after the vert end marker.  Then 6 other ones?
Next is what I'm calling a New Mesh Indicator, but probably has some other function: 3x 00 00 80 3F (read as 1.0 as a float?)
Next long is the address pointer to the of end of the mesh total (eg. 74 05 00 00 -> 0x0574)
then a null long, then a set of bytes that are a consistent header within the file but I haven't sorted them out yet (eg: 17 25 06 00   A3 9E 9C FF   3F 3F 3F FF   99 99 99 0C   40 00)
(somewhat consistent between files, at least the same header format, read as:  17 25 06 00   XX XX XX XX   XX XX XX XX   XX XX XX XX   XX XX.  )

Then an unsigned int short for....something? doesn't seem to line up with number of faces, or a pointer, or anything like that, but it changes for each mesh in the file.  It's bigger than the vertex count, but that's about all I know.  Maybe a total number of objects to process, and you subtract out the verts?

Then a mix of triangular faces and normals, maybe?  Triangular Faces using unsigned int shorts as vertex identifiers (I think), and probably normal vectors?  Not really sure how to break through this, the faces don't really align with anything resembling sensible geometry.  Also, there's periodically very large shorts, like FB FF, that break the vert indexing.  They one's complement into somewhat sensible vertex indices, but no other rhyme or reason as far as I can tell, and I don't know how to do that automatically.

At the end of the faces section, there's a long of FF 00 00 00, which I think is an end marker.

Then another short 29 00, then another short that's different for each mesh, then another null short.

Then a Vertex Count Short (unsigned int).  This one I'm fairly confident in, and it's reproduceable.

Then starts a mix of Verts and UV and maybe something else.  I don't really know if this is right, but it makes the address math work out:
each point is 6 floats: ??, U?, V?, X, Y, Z. 

Then after vertex count of verts, another FF 00 00 00 end marker. 

Next is 6 longs, and the first is the one that the End of Verts pointer addressed
Finally either a short and a null short, or a long, but a low int 24 00 00 00, or 36 00 00 00, and is the address of the end of mesh pointer.  Maybe it's actually the beginning of the next mesh?

Sometimes there's an empty mesh at the beginning or end.  Not sure, but they don't have an end of mesh pointer; sometimes they have a few vertexes, sometimes not.

I'll get to textures eventually, but I'd really like to figure out what the mesh format is first.

Anyhow, thanks for taking a read!  I don't really know what I'm doing yet, but I'd really appreciate any guidance you've got on this.

 

GSG03.7z

 

Edited October 1 by ScornMandark

[shak-otay]

Found gundam 3D formats always a little bit annoying. Small meshes wiht face indices not fitting. Something like that.

Anyways, here's some result using a point cloud skinner:

GSG03, one sub mesh

[ScornMandark]

Interesting!  There are some almost recognizeable shapes, but still not sure where it's going lol  

It looks like some of the map files, at least, are in .nj format, with the correct ninja chunk model format as read in by noesis.  On a hunch, I wondered if the robot mesh files were a version of that format, and we might be making some progress.  The sega ninja chunk model (njcm) format is somewhat well documented, as it was part of the dreamcast sdk, and the company that made this game worked on a dreamcast game previously.  Some things I had right, some things I was very wrong.  I'm using a .nj file import document that was based around phantasy star online for the definitions, here: https://bit.ly/PSOFile

NJCM Header:  

Typically has the NJCM header (0x 4E 4A 43 4D in little endian), followed by 4 bytes of flags.  These appears to be stripped out of most of the .md robot files, but still in the ,nj map files.

1x 4 bytes to id the model.
3x 4 bytes for model position as floats
3x 4 bytes for model rotation as Int32 (later normalized with 65535.0/360.0)
3x 4 bytes for model scale as floats (this is the 1.0 1.0 1.0 part of the header for all these files)
1x 4 bytes for child pointer
1x 4 bytes for sibling buffer

Vertex Def: very different to what I was expecting!

Vertsection Header: all int32

a_dword
list
b_dword
num_verts

Vertex:

x float
y float
z float

norm x float
norm y float
norm z float

color int32

u float
v float

 

then faces are vstrips

 

I don't have the vert indexes lining up right yet, but that would explain a lot.  Maybe I can trick noesis into thinking these are nj files and have it decompose them automagically

 

[roocker666]

I got this with other submesh, It looks like the head but I don't see normals or vcol. Just vertices and Uvs, I don't know if this is correct.. Yeah it looks similar to .NJ format, that is crazy.

Head verts & Uvs

 

[ScornMandark]

Nice!  Yeah, that looks like a head, nice to see some recognizeable shapes.  I found a better document for the NJ format, since the PSO njcm was expanded: https://bit.ly/NJFormat

I think it's a version of a simpler nj file - looking at the PSO format, the vert position, normal, color, and uv assignment should all be in the vert list, but I think it's only the xyz and normal xyz.  Plus, the vert list header is supposed to be 4x 4 bytes, and it's only 2x 4 bytes - or, more likely, 4x 2 bytes (shorts).  The vert list is addressed specifically by a pointer in the model header at the end of the mesh declaration, it's the first 4 byte chunk in that header.  It's only 8 bytes to the beginning of the point array, but the last 2 bytes very neatly count the number of verts, as long as it's just vert xyz and normal xyz.

Interestingly, noesis can read the amp01.nj file as an njcm, even with the modifications, so there may be some format extensions it handles already.  Just dumping the NJCM header up top didn't do it, but I'll see if I can't keep poking at it.

The model section looks like it skips any dwords, goes straight to the vert list as a long int, followed by a pointer to the v-strip list header for faces as a long int. (there's a hi res model file that's really big, and the addresses get up to 0x00 0f a4 90, which requires a long, and verified by looking at the last mesh in that file).  Then 4 floats for model center xyz and radius.

(the map file has an additional 5 x 4 bytes at the end compared to the multi-mesh model files, so that's probably something to do with it as well.)

 

I've attached the AMP01,NJ file, noesis can read that one directly for example.

 

AMP01.zip

[ScornMandark]

Ooooooooooooooooooooh ok, making a little progress. I found a repo of the dreamcast Katana SDK online, specifically the ninjacnk.h header which should decrypt many of the questions  (here).  Interestingly, there is an AMP01.AT file, which is very similar in structure to the .NJ file, but with some extra verts and faces.  The format is almost identical otherwise.

Object Header
This is the header at the very beginning of the mesh object definition:

/*
 * NJS_CNK_OBJECT
 */
typedef struct cnkobj {
    Uint32          evalflags;  /* evalation flags              */
    NJS_CNK_MODEL   *model;     /* model data pointer           */
    Float           pos[3];     /* translation                  */
    Angle           ang[3];     /* rotation                     */
    Float           scl[3];     /* scaling                      */
    struct cnkobj   *child;     /* child object                 */
    struct cnkobj   *sibling;   /* sibling object               */
} NJS_CNK_OBJECT;

So for now we are ignoring the eval flags, but the model data pointer is definitely correct, and the position/rotation/scaling/child/sibling objects all make sense so far.

 

Model Header
We get pointed here from the cnkobj struct, so let's look at this next:

 * NJS_CNK_MODEL
 */
typedef struct {
    Sint32            *vlist;   /* vertex list                  */
    Sint16            *plist;   /* polygon list                 */
    NJS_POINT3        center;   /* model center                 */
    Float                  r;   /* radius                       */
} NJS_CNK_MODEL;

This tells gives us a pointer to the vertex list (check), a pointer to the polygon list (check), the model center (3x floats) and the radius of the model.

 

Vertex Headers
Since the vertex pointer is first in the NJS_CNK_Model struct, the vertex header is set up like this (I snipped out a lot of extra definitions, so relevant info only):

#define NJD_VERTOFF            32 /* chunk vertex offset (32 bits size)   */

/*--------------*/
/* Chunk Vertex */
/*--------------*/
/* <Format>=[headbits(8)|ChunkHead(8)]                                    */
/*          [Size(16)][IndexOffset(16)][nbIndices(16)]                    */
/*     <headbits>(NF only)                                                */
/*        9- 8 = WeightStatus(2) Start, Middle, End                       */
/* VN  : use vertex normal                                                */
/* VNX : 32bits vertex normal  reserved(2)|x(10)|y(10)|z(10)              */
/* SH  : SH4 optimize                                                     */

  /* snip */
 
/* optimize for SH4 */
#define NJD_CV_SH     (NJD_VERTOFF+0)  /* x,y,z,1.0F, ...                 */
#define NJD_CV_VN_SH  (NJD_VERTOFF+1)  /* x,y,z,1.0F,nx,ny,nz,0.0F,...    */

/* chunk vertex */
#define NJD_CV        (NJD_VERTOFF+2)  /* x,y,z, ...                      */

  /* snip */
 
#define NJD_CV_VN     (NJD_VERTOFF+9)  /* x,y,z,nx,ny,nz, ...             */

  /* snip */

 

So the head bits are an 8 bit short, which start as 32 (0x 20), then add a value to that to set the definition.  The x,y,z,nx,ny,nz vertex definition adds 9, for 41 (0x 29), which is how every vertex group starts that I've seen so far.  Nice to have a little confirmation! 

Then 00 for the ChunkHead, 09 61 for the size (as a half float, that comes out to 644.5; not really sure what it's used for yet...), 00 00 for the IndexOffset (start the vert table right away), then 2C 10 for the number of indices (4140, which sounds right for the map file AMP01.nj).  So we know for sure there's 4140 verts, set up as x,y,z,nx,ny,nz.  And using that setup in ModelResearcherPro, we see that it lines up precisely with the end stop bit for the verts.  That's a pretty solid start!  It's also very consistent between all the different files that I've checked it against, so I think we can pretty solidly call that solved.  Woot!  The size value divided by the number of verts comes out to 6 (remainder 1), which suggests the size is referring to the size of the vertex chunks block, since 6 floats make up each vertex?  The AMP01.AT file also has the size as C3 61, with 4B 10 verts, and the division comes out the same.  The other model files I checked also have that info - I guess I think that's techincally doubled info, since it's already specified with the header, but at least we know what it is.

 

Polygon Header
Here's the tricky bit now, trying to get the faces.  Back to the PSO format analysis here, they found that the NJ use triangle strips for face declarations, and laid out in chunks.  Plus, they figured out that the header was a set of flags, then material diffuse (RGBA), ambient  (RGBA), and Specular (????), followed by chunk def.  Each vert strip was preceeded by a length, so you'd have one int for length, followed by the array, followed by the next int, etc, until the end of the chunk.

The problem is that none of the files line up that nicely 😛

but!  I think I may have come to a bit of a revelation.  Looking at GSG03.MD, the first length chunk doesn't go all the way to the end.  It does, however, go right up to what looks like another tex file header or flag.  Here's what I'm thinking - each mesh may be broken into several tex pieces, like if you've got multiple textures for a single object, or maybe just different tex groups.  In this case, we'd have group 1 defined by the first header - it has the material/texture flags at the front (1 & 2), followed by 3 longs for material properties (3, 4, 5), followed by an object type flag (6), then a chunk size (7), followed by the first chunk's number of vert strips (8), then the first vert strip number of verts (9), then the 4 verts defined by 9 (A,A,A,A).  Then, since all 4 verts were declared, the next vert strip size is declared (B), and the 4 verts therein are called (C,C,C,C).  Then there's more until the full size as defined by 7 is reached.  After that, a new chunk is defined with a set of material/tex flags (D,E), then only 2 longs for material properties (F,F), then Object type flag (G), then size of chunk (H), then number of strips in the new chunk (I), then Strip 1 length (J), then strip 1 verts (K,K,K,K), strip 2 (L,M,M,M,M), and strip 3 (N,O,O,O,O).  Finally, the stop var (P).  So far, I think it all works out really quite elegantly.  I can't tell the Model Researcher to split up vert strips like that, but at least I think I know what's happening!

 

 

And, by finding longer strips, multiple meshes with the same verts but different face strips, I can make model researcher create some of the parts of the object:

Some of the strip lengths are negative values for some reason - I wonder if those are stripR vs stripL?  Apparently it changes how the vert strips are wound, or unwound, or whatever.  That would make sense, though!  Well that's exciting - just need to figure out how to code all that into an exporter LOL

[ScornMandark]

Ok, pseudo code for translation: 

make 4 containers: 
    object (bone root, next object pointer, basic bone structure created thru that) 
    model  (main pointers, object center)
    vertices (array of vertices and normals) 
    polygons (array(s) of vertex strips, may be multiple sections)
    

First container is for the first object, and no sense reinventing the wheel. Should be consistently 13x4 byes, at least

/*
 * NJS_CNK_OBJECT
 */
typedef struct cnkobj {
    Uint32          evalflags;  /* evalation flags, ignore for now    */
    NJS_CNK_MODEL   *model;     /* model data pointer for this object */
    Float           pos[3];     /* translation   bone                */
    Angle           ang[3];     /* rotation      bone                */
    Float           scl[3];     /* scaling       bone               */
    struct cnkobj   *child;     /* child object  this is typically the next one in the file */
    struct cnkobj   *sibling;   /* sibling object               */
} NJS_CNK_OBJECT;

It bounces us to the end of the object for the model definition.  6x4 bytes, although I dunno how much shifting on the root bone happens.

* NJS_CNK_MODEL
 */
typedef struct {
    Sint32            *vlist;   /* vertex list                  */
    Sint16            *plist;   /* polygon list                 */
    NJS_POINT3        center;   /* model center                 */
    Float                  r;   /* radius                       */
} NJS_CNK_MODEL;

Then the verts.  Header is consistent at 4x2 bytes, and size of vert array is fixed. 

/*--------------*/
/* Chunk Vertex */
/*--------------*/
/* <Format>=[headbits(8)|ChunkHead(8)]                                    */
/*          [Size(16)][IndexOffset(16)][nbIndices(16)]                    */
  typedef struct verthead {
      Uint8         VertChunkHeadBits; /*should basically always equal 29 */
      Uint8         VertChunkHead;     /*basically always 00 */
      Uint16        VertChunkSize;     /*can be used to verify array size */
      Uint16        VertChunkIndexOffset;   /* start reading verts right after header */
      Uint16        VertArraySize;     /* number of vert entries  */
  } MD_VERT_HEAD;
  
  typedef struct vert {
      float         XPos;              /* vert x position */
  
  } MD_VERT;

 

[ScornMandark]

Should have broken that up more lol.  Ok, verts again since I apparently can't delete code blocks on my phone lol

/*--------------*/
/* Chunk Vertex */
/*--------------*/
/* <Format>=[headbits(8)|ChunkHead(8)]                                    */
/*          [Size(16)][IndexOffset(16)][nbIndices(16)]                    */
  typedef struct verthead {
      Uint8         VertChunkHeadBits; /*should basically always equal 29 */
      Uint8         VertChunkHead;     /*basically always 00 */
      Uint16        VertChunkSize;     /*can be used to verify array size */
      Uint16        VertChunkIndexOffset;   /* start reading verts right after header */
      Uint16        VertArraySize;     /* number of vert entries  */
  } MD_VERT_HEAD;
  
  typedef struct vert {
      float         XPos;              /* vert x position */
      float         YPos;              /* vert y position */
      float         ZPos;              /* vert z position */
      float         XNorm;             /* vert x normal   */
      float         YNorm;             /* vert y normal   */
      float         ZNorm;             /* vert z normal   */
  
  } MD_VERT;

I guess we need to actually read some data.  Note to self, the cnkmod struct needs changed, the *plist is Sint32.

cnkobj object
start *pointer at 00 00 00 00
read 13x 32 byes, fill object.whatever

move *pointer to object.*model

cnkmod model
read 6x 32 bytes, fill model.whatever

 

Then we'll need to create the vert array and fill it.

vert vertArray[verthead.VertArraySize]

for(i=0;i<verthead.VertArraySize;i++){
                                       start pointer at cnkmod.*vlist+64 (size of verthead)
                                        read 6 floats 
                                        assign to vertArray[i.XPos,i.YPos,i.ZPos,i.XNorm,i.YNorm,i.ZNorm]
                                       move pointer 96 (size of vert)
                                       }
 verify next long is 0x FF 00 00 00 
                                       
 

 

[ScornMandark]

Maybe we should start by defining the model structure? Read the first object, define as root, then check for a child pointer. 

If there's a child, that object becomes a child of the root.  If there's a sibling object, it is also a child of the root. 

If there's a child of a child object, that object becomes a child of that child.  If there's a sibling object, it is also a child of that child object.  

Continue until all the child objects have no children, and all siblings exist. 

Then go through model definitions (assigned to specific objects), vert arrays, and faces. Yeah?  That way we can be sure we've got all the parts accounted for, and we can use object level transformations for vert arrays.

[ScornMandark]

Ok, now that we've got one object.model in particular, time to create the face array.

This one feels a lot more complicated

/*--------------*/
/* Chunk Polygon */
/*--------------*/
/* <Format>=[headbits(16)|ChunkHead(16)]                                    */
/*          [Diffuse(32)][Ambient(32)][Specular(32)][ModelType?(16)][nbStrips(16)]                    */
  typedef struct polyhead {
      Uint16        PolyChunkHeadBits; /* often 17 25 or 15 25 */
      Uint16        PolyChunkHead;     /* often 06 00 or 04 00 */
      Uint32        PolyChunkDiffuse;  /* actually 4x Uint8 */
      Uint32        PolyChunkAmbient;  /* actually 4x Uint8 */
      Uint32        PolyChunkSpecular; /* actually 4x Uint8 */
      Uint16        PolyChunkType;     /* model type i think? */
      Uint16        PolyChunkSize;     /* size of this array in bytes */
      Uint16        PolyChunknbStrips; /* number of vert strips in this chunk */
  
  } MD_POLY_HEAD;
 

Then we read the header and poly arrays

polyhead polyheadarray[]
move *pointer to cnkmod.*plist
local int j =0
while(true) {
    if j==0, readbytes=160, else readbytes =128
    read readbytes bytes, assign to polyheadarray.j.whatevers (set diffuse to 00 00 00 00 if j>0)
    if (*pointer + polyheadarray.i.PolyChunkSize == 0x FF 00 00 00) break, 
    else *pointer += polyheadarray.i.PolyChunkSize, j++
}
move pointer to cnkmod.*plist+160
for (i=0; i<j; i++){
    int[] polyheadarray.j.polystriparray[] /* array of arrays */
    nbStrips = polyheadarray.j.PolyChunknbStrips
    for (s=0; s<nbStrips;s++)
    {
        local length = read Sint16, pointer+16
        if length<0, is StripR, length = length *(-1)
        for (l=0; l<length-2; l++){
           polyheadarray.j.polystriparray[l,l+1,l+2] = *pointer, *pointer+16, *pointer+32
           *pointer+=32
        }
     }
     *pointer+=128
}
        

Now we've got arrays of triangles referenced by vertexes, broken into what I assume are material groups? 

Then find the first unfilled bone after the root and repeat 

[ScornMandark]

Maybe set the poly array material def based on the flags?  I'll need to check some of the other ms files

[ScornMandark]

Ok, so the header for the poly array is 17 25, or 15 25.  Based on the chunk file definition, the chunk header would be the 25, and the chunk type bits are the 17.  17 is between 16 and 31, which tells me it's a chunk material type.

/*======================================*/
/*                                      */
/*      Chunk Table                     */
/*                                      */
/*======================================*/
/* Chunk type offset */
#define NJD_NULLOFF             0 /* null chunk (16 bits size)            */
#define NJD_BITSOFF             1 /* chunk bits offset (16 bits size)     */
#define NJD_TINYOFF             8 /* chunk tiny offset (32 bits size)     */
#define NJD_MATOFF             16 /* chunk material offset (32 bits size) */
#define NJD_VERTOFF            32 /* chunk vertex offset (32 bits size)   */
#define NJD_VOLOFF             56 /* chunk volume offset (32 bits size)   */
#define NJD_STRIPOFF           64 /* chunk strip offset                   */
#define NJD_ENDOFF            255 /* end chunk offset (16 bits size)      */

More specifically, matoff+1, which is the Diffuse (alpha RGB) material type.  Actually NJM_CM_DAS, see below.

/*----------------*/
/* Chunk Material */
/*----------------*/
/* <Format>=[ChunkHead][Size][Data]                        */
/*       13-11 = SRC Alpha Instruction(3)                  */
/*       10- 8 = DST Alpha Instruction(3)                  */
/* D  : Diffuse (ARGB)                            bit 0    */
/* A  : Ambient (RGB)                             bit 1    */
/* S  : Specular(ERGB) E:exponent(5) range:0-16   bit 2    */
#define NJD_CM_D    (NJD_MATOFF+1)  /* [CHead][4(Size)][ARGB]

 

If I'm reading it right, it looks like the 0x25 is from the chunk bits defining the alpha. 

0x25 (bits 15-8) is 00100101.  

If Source Blend is Source Alpha (FBS_SA), then bits 13-11 are 100.

If Destination Blend is Inverse Source Alpha (FBD_ISA), then bits 10-8 are 101.

00 100 101 is 0x25

Nice!

 

Edited October 7 by ScornMandark
can't keep hex and dec straight

[ScornMandark]

No, no, no.  17 is 0x17, goofus, which is actually 23.  That's matoff+7, which is NJD_CM_DAS - Diffuse, Ambient, Specular.

#define NJD_CM_DAS  (NJD_MATOFF+7)  /* [CHead][12(Size)][ARGB][NRGB][ERGB] */

Plus, the 6 defines it as short size, so now we're making a lot more sense. 

Double bonus, some of the headers are 15 25 04 00.  With my renewed understanding that hex is not dec, that means it's matoff+5, which is NJD_CM_DS - Diffuse Specular, Ambient no change.  Size 4, and only 2 of the three material properties changed, explaining why one of the 32 bit values was left off.

 

Ok, I think I'm more or less up to speed on what it all means, just gotta figure out why I can't make noesis take it and run with it.

[ScornMandark]

OK, so now we know the 0x 17 25 06 00 is the beginning of the material flag, and goes for a total of 16 bytes.  Following that, it must be the chunk strip flags.  For GSG03.MD, the first chunk mat ends at 0x0044, so that means the chunk strip must begin there.  It starts with 0x 40 00 42 00 0C 00.

Header before the data is 3x 16 bit values.  Couple things go into it:

the ChunkHead is what sets the type of chunk strip we're looking at.  The plain Chunk Strip format (NJD_CS) is just the index of verts that make up the face strips, and has no other info on the faces, and the chunk head should be NJD_STRIPOFF+0.  At 64, that would be read as 40.  What do you know, nice. 

There looks like there's no chunk flags, then, all the mat info is in the mat header. 

Size is 0x 42 00, or 66 shorts until the next chunk. That one took me a minute to figure out, but it works if you include the nbStrip short in the count.

Number of strips is 0x 0C 00, or 12 strips, and the user offset flag must be 0, so no user offset flags between strips.  Count carefully, my friends.

If the flag in the length array is set, then it is a StripR format, otherwise it's StripL.  I don't think it actually matters in terms of reconverting, but I could be wrong.

/*-------------*/
/* Chunk Strip */
/*-------------*/
/* UserFlags N=0,1(16bit*1),2(16bit*2),3(16bit*3)                         */
/* <Format1>=[CFlags(8)|CHead(8)][Size(16)][UserOffset(2)|nbStrip(14)]    */
/*           [flag(1)|len(15), index0(16), index1(16),                    */
/*            index2, UserFlag2(*N), ...]                                 */

This is the first strip array in the file.  0x 000048 says there should be 12 strips, 0x 00004a is where the first length var should start, and it looks like 4.  I'm going to put my interpetation below:

I don't know why there are 13 strips for 12 declared, but using 13 strips matches up with the 66 short declared size of the chunk, including the nb strips.  Maybe they start at 0?  Typing tired makes sloppy compiling, 12 strips = 12 strips.

[ Strip L, Length 4 (2 triangles)
  V21, V45, V16
       V45, V16, V41  ] /* S01 */

[ Strip L, Length 4
  V36, V42, V12
       V42, V12, V19  ] /* S02 */

[ Strip L, Length 3 (1 triangle)
  V34, V32, V33  ] /* S03 */

[ Strip L, Length 4
  V39, V37, V42
       V37, V42, V44  ] /* S04 */

[ Strip L, Length 3
  V22, V25, V27  ] /* S05 */

[ Strip L, Length 4
  V20, V17, V13
       V17, V13, V14  ] /* S06 */

[ Strip L, Length 4
  V38, V32, V40
       V32, V40, V35  ] /* S07 */

[ Strip L, Length 4
  V15, V18, V24
       V18, V24, V26  ] /* S08 */

[ Strip L, Length 3
  V6, V4, V5  ] /* S09 */

[ Strip L, Length 11 (9 faces)
  V10, V9, V8
       V9, V8, V5
		   V8, V5, V7
 			   V5, V7, V4
			       V7, V4, V11
			           V4, V11, V6
  			               V11, V6, V10
  				                V6, V10, V5
			                        V10, V5, V9 ] /* S10 */

[ Strip L, Length 4
  V10, V8, V11
       V8, V11, V7  ]  /* S11 */

[ Strip R, Length 5 (3 faces)
  V3, V0, V1
      V0, V1, V2
          V1, V2, V3  ]  /* S12 */

Following that chunk strip, there's another material header.  0x 15 25 04 00 00 00 00 FF 00 00 00 0C means NJM_CM_DS, as in only update the diffuse and specular fields, which are both basically turned off, I guess, based on the values?  The 04 then specifies the 4 shorts worth of data, which tracks.

Then another chunk header, 0x 40 00 10 00 - plain Chunk Strip, no flags, and 16 shorts to the next chunk.  also tracks, since 16 shorts ahead is the strip stop byte marker.  3 strips as follows:

[ Strip L, Length 4
  V23, V28, V29
       V28, V29, V30 ]  /* strip 1 */

[ Strip L, Length 4
  V33, V32, V37
       V32, V37, V39  ]  /* strip 2 */

[ Strip L, Length 4
  V24, V13, V41
       V13, V41, V14  ]  /* strip 3 */

This one was actually 3 strips, so I dunno anymore what is going on with the strip count.  Maybe just do it based on byte size?  That seems risky lol  Maybe create an array of size nbStrips, but just check at the end to see if you hit your byte address marker, and read another strip if you're not there?

Edit: miscounted, there are only 12 strips, I'll re-edit the code block tomorrow.
 

Edited October 8 by ScornMandark
updating hand strip counting

[ScornMandark]

ok, with this newfound understanding, let's get pedantic 😛

looking at the BUGU03.MD file (guaranteed to be the CCMS-03 BUGU), the first object is made up of 3 different material chunks, with 18 strips between them.  (One of the strips ended on 0x 6D, and it looks like it got padded out by a null byte to make the next chunk start on a nicer address since that's how the chunk length short was described...)  It's pretty recognizable as a hip joint! with the front center connector, back plate, and hip core.  That's some pretty exciting nonsense there LOL 

 

Interestingly, there are some nearly identical files included for several of the MS.  The J Saviour, for example, has I think 9 different files associated with it.  The JSG_TX.GSP and JS01M01.GSP are almost identical in size, and virtually identical in content.  Same for JSG_TX.MD and JS01M01.MD, JSG02.MD and JSN02.MD, and JSG03.MD and JSN03.MD.  The only outlier is the JS01M01.MTP, which I think is the animation file.

[ScornMandark]

Well hello there.

[ScornMandark]

On 10/2/2024 at 4:53 PM, roocker666 said:

I got this with other submesh, It looks like the head but I don't see normals or vcol. Just vertices and Uvs, I don't know if this is correct.. Yeah it looks similar to .NJ format, that is crazy.

Head verts & Uvs

 

that's it exactly, it's the head mesh.  I exported it to obj and noesis deemed it acceptable.  

next up is decoding the texture files, and writing a script so I don't need to do all that by hand every time 😛

GSG03_head.7z

[ScornMandark]

Welp, that's a whole thing!  I just inserted 8 bytes into the file - the NJCM header, 4 bytes describing the file lenght, and a null 4 bytes, and Noesis happily returned this glorious PS2 chunky fella

 

Just need to sort out how the texture files can be done the same way

[ScornMandark]

Which means they're all crackable now!

 

[ScornMandark]

The in game models aren't quite as high poly, but they'll do.  They clearly had the high res models for the game manual and splash screens, it would have been awesome if they'd shown a different model at the start screen periodically.  Awesome for me, that is 😛

 

Still, on to the textures.  It looks like the material definitions in the model files are generally ARGB diffuse, NRGB ambient, ERGB specular.  I'm going to try some of the texture explorers and see if I can stumble into them, hopefully I'll be able to export some UV info from them as well.   There's .GS files and .GSP files, as well as .mt and .mtp.  it looks like the .**p files include the back end of a header of some kind, starting with the number of 4-byte chunks until the dividing header, followed by a GS tag.  It looks like the dividing header is to align the GS header to the start of a 16 byte line (0x0FF0, 0x43D0, etc).

The bytes are filled for a decent chunk, then seemingly empty, then filled for a longer chunk.  There's probably a pattern to it, I'm just not sure what yet.  

They've got to include the uv info as well, I think - the model files don't include any mapping, so there's got to be a key there as well.

[The_oldman]

I did follow your progress and you are always advance me.

now I have question. Where are the UVs. If .md is just .nj files without the header 8 bytes then there will be no UV!?

And I am sure .gsp is images file. 

first 4 byte is image count. then follows each image size then padding for 0x10

in the image section

Byte 00-01 2 bytes header(GS)

byte 02-03 2 byte(short) image type (always 0x13: 8-bits Index per pixel)

byte 04-05 image width

byte 06-07 image height

byte 08-0F padding

then Palette section 4 bytes each color (256 color for type 0x13)

last Pixel section  

 

Here is the first cut of gsg01m01.gsp image

gsg01m01-000.rar

[ScornMandark]

39 minutes ago, The_oldman said:

I did follow your progress and you are always advance me.

now I have question. Where are the UVs. If .md is just .nj files without the header 8 bytes then there will be no UV!?

And I am sure .gsp is images file. 

first 4 byte is image count. then follows each image size then padding for 0x10

in the image section

Byte 00-01 2 bytes header(GS)

byte 02-03 2 byte(short) image type (always 0x13: 8-bits Index per pixel)

byte 04-05 image width

byte 06-07 image height

byte 08-0F padding

then Palette section 4 bytes each color (256 color for type 0x13)

last Pixel section  

 

Here is the first cut of gsg01m01.gsp image

gsg01m01-000.rar 2.22 kB · 0 downloads

Legend!  Oh man, that helps a ton!  I ended up using pixelDBG and checked around a bit, and it worked a treat ❤️ 

So the gsg01m01.gsp has 88 textures, and the first one is 0x1410 long.  The header starts GS, followed by 0x 00 13, which definitely looks like a image type, and 8 bits/color/pixel.  (There are some files with 0x 00 14, so we'll need to verify how that changes.)  Image width and height check out, and the pallette has 256 entries for each color, check!

I began the actual image offset at 0x590, and it's lovely!

I got greedy, and I wanted to see bigger images, so I went for the GSS_HI.gsp.  Interestingly there's several 0x0014 and 0x0002 image types, so I'm not sure what's going on with those just yet.  Still, there are several quite nice ones already:

 

 

 

The 0x0002 header has a length of 2576 bytes, and is marked (ostensibly) as a 32x32-pixel image.  That's more than twice the bytes, though, assuming it's still 4-bytes per pixel!  (1024 pixels x 4 bytes is 4096 bytes.)  In the buffer zone behind the LxW info, there's a 0x0001 where it's usually just zeroes.  Not sure what it means, or the 02 instead of 13, but something interesting.

On to the UVs though, I still have no idea where to look for those.  The only other file that's typically associated with these models are a .mtp, which is pretty similar looking to nmdm file, kinda:

4 bytes: number of anims (n)
n*4 bytes/each: length of each anim in bytes, sequentially
(n+1)*4 bytes: number of animation frames (4 bytes, x)
(n+5)*4 bytes: 2x 16 bit values, maybe motion type?  bone number?
(n+9)*4 bytes - (n+9+x)*4 bytes: keyframe (int), x, y, z rotation (3x float, or int then 66536/360 deg)

 

[ScornMandark]

based on the NJS_Motion spec, that is set up as:

typedef struct {
void *mdata;		/* Array for object tree*/
Uint32 nbFrame;		/* Number of motion frames*/
Uint16 type; 		/* Motion element bit string*/
Uint16 inp_fn;		/* Interpolation method and number of elements*/
}NJS_MOTION

The first one is there, but I don't really know how to interpret it; the number of frames seems right (if it's the number of frames for that animation); the type (7) would indicate bits 0,1,2 are set, and so position, angle, and scale are all called (not sure this is right, tho).  Interpolation method of linear and element #3 indicates that whatever this animation is, element 3 of the main model structure is being modified?

Followed by the keyframe number (4 bytes) and then 12 bytes.  If the position and vectors are set as floats, and angles are Sint32, then it can't be all 3 kinds of keyframe data.  Probably just the 3 axis angles, then, is my guess, but I could be wrong. 

For GS01M01.MTP, the first array pointer offset 0x 174C neatly points to the end of the animation data, which isn't the end of the motion byte chunk described by the top header.  There's some random data in the void space, but who knows how to interpret it.  Maybe it's a sequential bone structure animation, implying each bone in sequence?  there's 12 sequences, I think there's more than 12 bones?  I dunno.

[ScornMandark]

Ninja ripper on PCSX2 gives me very similar textures in very similar sizes and formats, so there are definitely options if crawling through the gsp file isn't as appealing.  Still no idea where to get UVs from, though.  NR has UV output, but the files are so distended in the rip that I don't think they're super useful.

Note, any .gs file is necessarily a single texture file, since there's not multi-file structure header like from the ,gsp.

Maybe 0x0014 is grayscale?  the 14 chunks are mostly 00  or 11 or 22, pretty limited range.

[The_oldman]

I don't have any file in hand right now. But I remember there is a image type use 4 bits per color Indices(0x14 maybe?). And the palette will only have 16 colors.

eg: 32 X 32 image with 512 bytes Indices and 64 bytes palette

I believe there is a full image type which is 32 bits per pixel without palette