[PS2] Gundam G-Saviour model file help

[ScornMandark]

Well now.

 

Apparently when Noesis converted the njcm, it also captured the UV mapping somehow.  I do not question this magic, I just accept that I don't know how it happened 😛 

[The_oldman]

6 hours ago, ScornMandark said:

Well now.

 

Apparently when Noesis converted the njcm, it also captured the UV mapping somehow.  I do not question this magic, I just accept that I don't know how it happened 😛 

Could you tell me how you achieve this in detail?

[ScornMandark]

Sure thing - 

1) Extract ISO of disk with 7-Zip or Winrar

2) in the Data folder, find the suit you're after.

• GSS_HI is the high res G-Saviour Space Mode
• Others are generally straightforward to figure out - GS for G Saviour, GSG for G Saviour Ground, JS for J Saviour, BG for Bugu, etc.

3) identify the biggest matched set of file names - GSS_HI.MD is the model, GSS_HI.GSP is the multiple-texture file.

4) For the model, open it in a hex editor and check the length of the file in bytes by finding the address of the byte after the last one

• for GSS_HI.MD, it was 0x0FA49C

5) inject 8 bytes into the header, keeping in mind the little endian-ness

• first 4 bytes are 4E 4A 43 4D (NJCM)
• second 4 bytes are the little endian version of the length.  For GSS_HI.MD, those would be 9C A4 0F 00

6) Save as <Filename>.nj

7) Open Noesis, open .nj, save as .fbx.  Model is done!

8 ) Either use ninjaripper to extract the scene textures, or go into the GSS_HI.GSP file with something like PixelDbg.

9) with PixelDbg, search for the GS text preceded by FF FF FF FF - that's the beginning of each new texture.

10) if the next 2 bytes are 13 00, then it's an 8.8.8.8 BGRA pixel setup.  the next 2 shorts are the length and width.  Should be square, but 03 00 03 00 means 48x48, etc.

11) the next full line (assuming 16 byte lines) starts the pallette.

12) 1024 bytes later starts the image.

13) export, and move to the next.  (this method has the advantage of keeping track of where in the file the image was, and noesis outputs the UV textures as njtex00, njtex01, etc, and they are in the same order as the textures).

14) open the fbx in blender, apply images as diffuse maps, reduce metallic and specular by half.

15) apply eye texture as emission map, put in a camera and a few lights, and render away!

16) the 14 00 texture indicator I think is a 2.2.2.2 BRGA, but it might also be a 8.0.0.0 grayscale, I'm not sure yet.

17) the other model files have dozens of textures, and it takes a long time to apply them all 😛 

 

 

Note: as I develop and improve the methods of ripping, I'll update this post!

Edited October 10 by ScornMandark

[ScornMandark]

Ok, so I'm probably just exhausted, but I'm having trouble wrapping my head around writing a noesis plugin.  I just want it to output a series of PNG files named njtex0xx, based on order in the <FileName>.GSP file, or just njtex000 if it's a standalone <FileName>.GS file.  Each .GS file, whether embedded in the GSP file or standalone, has the same format:

2 byte header: 0x 47 53 ( G S )

2 byte type int: 0x 13 00 (imgType=b8g8r8a8, textByteSize=4, palletSize=0x10 - 0x400 bytes of pallette, 4 bytes per pixel, 0x100 pixels in pallette.  0x10 x 0x10 size)

                           0x 14 00 (imgType=8.0.0.0 grayscale?, texByteSize=1, palletSize=8 - 0x40 bytes of pallette, 1 byte per pixel, 0x40 pixels in pallette. 0x08 x 0x08 size)

(Might be others, but these are the primary two.)

1 byte int pixel width

1 byte int pixel height

(not always square, but typically. Not always power of 2 either - 8, 16, 24, 32, 48, 64, 128, 256, 480, 512 all seen for both values)

4 bytes padding: 0x 00 00 00 00 (I've seen an occasional 0x 01 00 00 00, but I dunno what it means)

palletSize*texByteSize x palletSize*texByteSize pallet

width*texByteSize x height*texByteSize image, imgType pixel data

That's it. GS standalone files end after the pixel data.

GSP files have no magic number header, just a 2 byte int for number of textures.  Followed by 4 byte ints for each texture length in bytes.  Then pad with 0x FF FF FF FF until the next 0xXXX0 address.  If the GSP header ends on a 0xXXX0 address, pad a full 16 bytes until the next 0xXXX0 address.

Right after the padding, always on a 0xXXX0 address, a standard GS file begins.  At the end of the GS file, pad until the next 0xXXX0 address.  Again, if the file ends on an aligned address, pad out until the next one.

Ideally they'd all get put into a folder called <FileName> in the same folder as the <FileName>.GS or .GSP file - that way it could batch process without overwriting other files.

Edited October 12 by ScornMandark
Adding in organizational requirements; corrected palette size math

[ScornMandark]

Maybe a two step process - batch convert all <FileName>.GSP into njtex0xx.gs files in a <FileName> subfolder.  Then, batch convert each .GS file into a .PNG file in place.  I think that makes it a much more manageable process.

[The_oldman]

sorry guys, I'd been too busy in real life and didn't come back here.

for GS texture, 0x14 is 8bits indices + 256 color palette, 0x13 is 4bits indices + 16 color palette and 0x02 is 16bits pixel without palette.

also after W and H, there is a short value for no. mipmaps.

I made two quickbms script to convert md to nj and gs/gsp to tm2  directly. hope it help! :)

G_Scripts.rar

Edited October 13 by The_oldman

[ScornMandark]

8 hours ago, The_oldman said:

sorry guys, I'd been too busy in real life and didn't come back here.

for GS texture, 0x14 is 8bits indices + 256 color palette, 0x13 is 4bits indices + 16 color palette and 0x02 is 16bits pixel without palette.

also after W and H, there is a short value for no. mipmaps.

I made two quickbms script to convert md to nj and gs/gsp to tm2  directly. hope it help! 🙂

G_Scripts.rar 1011 B · 2 downloads

Oh snap!! That's awesome, man! Thanks so much!  Tomorrow I'm gonna blast this at the archive and see what survives hahahahaha

[ScornMandark]

19 hours ago, The_oldman said:

sorry guys, I'd been too busy in real life and didn't come back here.

for GS texture, 0x14 is 8bits indices + 256 color palette, 0x13 is 4bits indices + 16 color palette and 0x02 is 16bits pixel without palette.

also after W and H, there is a short value for no. mipmaps.

I made two quickbms script to convert md to nj and gs/gsp to tm2  directly. hope it help! 🙂

G_Scripts.rar 1011 B · 3 downloads

OK, .md to .nj worked a treat!  so far so good, that is frikkin awesome.  Having an odd issue with the GSP2TM2 - when I'm using pixelDbg's native bmp export, the output is a lot smoother than with the tm2 output:

Is that just the TM2 format compressing it?  If we could output to .png or .tga natively, that would be a great step.  Or maybe it's just the BMP ignoring the alpha channel?

I also didn't break it too badly updating the output naming convention, which I was quite pleased with lol:

set NINJA "njtex"
string RNAME p "%s/%s%03d.tm2" NAMEBASE NINJA i

This will output the textures into a named subfolder with the naming convention that Noesis generates for the textures, making texture assigning much faster for both noesis and blender.

This is texture 2 (njtex001) from the attached JS01M01.GSP

  

 

GSP2TIM_mod.zip

JS01M01.zip

Edited October 14 by ScornMandark

[ScornMandark]

The BMS script matches well to Console Texture Explorer, but PixelDbg still has a smoother output.  The output from the pixelDbg is a 24-bit RGB (no alpha) bitmap, using the BITMAPINFOHEADER (40 bits), no compression, 24-bit rgb, no palette.  

 

 BT01M01.GSP_64x64_4816.bmp.zip

 

Iiiiinterestingly, if I select the PSP console in the console texture explorer, it comes up with a properly smoothed out version:

 

so how does psp look at it different than ps2?

Edited October 14 by ScornMandark

[ScornMandark]

It looks like it basically all comes down to the palette - I don't really know what the difference is, but both the .bmp and .tm2 outputs for the psp and ps2 versions have identical indexing, just different palette sections.  (bmp palette starts up at 0x56 for both files, and the tm2 palette starts at 0x1040 for both.)  (The output tm2 files for both CTE and the scriptare nearly identical).  Many of the palette entries are the same, but they're not all in the same order.

Being an engineer, you know a spreadsheet had to enter the chat.  I grouped the hex values by 4 byte chunks, imported them into excel, then used a formula to match PSP palette index locations to PS2 palette index locations.  Every 8 palette entries looks like a block, and for the PSP (and the GSP files), the first block is regular, then the 2 sets of palette index blocks are swapped relative to the PS2, then the next 2 are regular, then swapped, etc.  Does this make any sense?  it doesn't really to me, as in why they would run it like that, but the results are real.

The GSP file texture block matches the PS2 texture block, but for whatever reason the PSP interpretation (and the PixelDbg interpretation) swap the second and third out of 4 blocks in the palette.

 

 

CTE.zip

Edited October 15 by ScornMandark
missed the first block of the palette, updated

[ScornMandark]

Ok, so if for some reason we treat 8x 4-byte palette entries as a block, that's 32 bytes of data at a chunk.  If they were conceptualized as a quad of chunks, 128 bytes at a time:

A - B - E - F

C - D - G - H 

I could see it getting read in 2 different ways.  One is a Z pattern - A B C D, E F G H.  This method is linear read of the palette file, which I assume the PS2 format is doing. The other is an upside down N - A C B D, E G F H.  I dunno why you would do it that way, but you could. That way you'd chunk the color lookup table for what I assume are memory reasons?  I still don't really understand why, but clearly it works.  32-byte clut buffer with swizzled middle chunks, but unswizzled pixel indexes.  Considering it was written for a PS2, and not a PSP...

[The_oldman]

Because of VRAM limitation most console use swizzled images in heavy graphic game. That why I use TM2(PS2 native) images for conversion. But this one seems didn't use standard TM2 method then!

I see what I can do later tonight.

[ScornMandark]

2 hours ago, The_oldman said:

Because of VRAM limitation most console use swizzled images in heavy graphic game. That why I use TM2(PS2 native) images for conversion. But this one seems didn't use standard TM2 method then!

I see what I can do later tonight.

I can't wait!  Thanks so much for your help, you've already helped get this like 90% of the way to being actually completed finally ❤️ 

[The_oldman]

OK It seem all gs images are linear format. They can be converted into bmp directly. So I make a quick and dirty BMS script to do these job. The converting process is a little bit slow so it convert single gs image once at a time! Also 0x02 type (5551 pixel format) is not supported since there are only a few. You can use other program like TextureFinder to convert them. 

how it works:

a. convert gsp into gs (using  GSPextract.bms)

b. convert gs to bmp (using  GS2BMP.bms)

there are some batch files to help you running QuickBMS in easy. Just edit the path of the batch files path will work. Extract_GSP_to_GS(Drag_and_Drop_Here).bat and Convert_GS_to_BMP(Drag_and_Drop_Here).bat can be in anywhere and as the name said drop the source file into it will do. if you need batch conversion, place Batch_Convert_GS_to_BMP.bat in the same folder as the gs files are and double click the .bat file

G_Scripts2.rar

 

PS: feel free to edit these scripts

Edited October 16 by The_oldman

[ScornMandark]

15 hours ago, The_oldman said:

OK It seem all gs images are linear format. They can be converted into bmp directly. So I make a quick and dirty BMS script to do these job. The converting process is a little bit slow so it convert single gs image once at a time! Also 0x02 type (5551 pixel format) is not supported since there are only a few. You can use other program like TextureFinder to convert them. 

how it works:

a. convert gsp into gs (using  GSPextract.bms)

b. convert gs to bmp (using  GS2BMP.bms)

there are some batch files to help you running QuickBMS in easy. Just edit the path of the batch files path will work. Extract_GSP_to_GS(Drag_and_Drop_Here).bat and Convert_GS_to_BMP(Drag_and_Drop_Here).bat can be in anywhere and as the name said drop the source file into it will do. if you need batch conversion, place Batch_Convert_GS_to_BMP.bat in the same folder as the gs files are and double click the .bat file

G_Scripts2.rar 1.4 kB · 2 downloads

 

PS: feel free to edit these scripts

Dude.  DUDE.  You are a legend amongst men, that is so frakkin' amazing.

There's some extra bits I'm going to try to modify to make bulk unpacking work out easier, but many, many thanks to you my friend ❤️ 

[ScornMandark]

Ran into an issue when scraping a different model - the offset padding for the 062 .gs file didn't align nicely, and it only had 5 bytes of padding instead of 16, so the last 5 textures didn't start with the GS magic word, so the bmp conversion failed for those.   Looking at the previous model, I think I may have run into the same issue there, actually.

Is there a way to adjust the padding based on the image offsets?   You can see the texture in question - all the other lengths end in 0x10 or 0x50 or what have you, but this one ends in 0xD9, which throws it all off.  You need to pad 7 to to get it aligned again, which takes the smallest byte value back to 0.  In C that's using something like modulo, like:

xmath tex_end "GS_SIZE % 16"  # that will keep even the full 16 byte padding for well aligned offsets.

xmath tex_pad "16 - tex_pad"

xmath OFS_BLK "OFS_BLK + GS_SIZE + tex_pad"

[ScornMandark]

Oh, that was a lot easier than I thought LOL This is so great, I really can't thank you enough for all your help!  I updated the script and attached it, and I updated the GS2BMP script to output the njtex named files like noesis creates from the nj files.  I also added a second GS2BMPBaseFolder script, which turns any gs files in the main Data folder into .bmps but in the named file subfolder.

 

 

GSaviourBMS.zip

Edited October 17 by ScornMandark
put the full set of BMS files in the zip folder

[ScornMandark]

OK, I set up a github repository to host the scripts, hopefully they'll help anyone who's curious about it!

 

https://github.com/ScornMandark/G-SaviourExtract

[ScornMandark]

I wrote my first blender script!  I updated the github repo with the AutomapBMP.py and instructions on how to use it.  In short, if you import the noesis FBX output file into blender, save it in the folder with all the export .bmp files, then run the script, it goes through and creates an image node, mapped to the same named njtexXXX.bmp as the material name, if it has a material slot (because I kept selecting the bones lol).  much, much less tedious than before.