Jump to content

[PS2] Tokimeki Memorial 3 - Getting Models from eeMemory?


Go to solution Solved by Natsuki Okusawa,

Recommended Posts

Would it be possible to get the models from eeMemory? It appears like that the game has compressed files for its assets, which are in those .atp files (I also made a separate thread about those files, but no reply so far...) so I'm thinking of the possibility of getting the models from eeMemory instead. Can anybody take a look?

eeMemory.rar

 

I suspect that the data that starts from 0x1700000 is a model data, but I'm not too sure.

0x1700000.thumb.png.3b90ffd108c82ab0475e422b1a0fca6a.png

I also pulled this data that I suspect to be a 3d model manually with a hex editor. Here is the file:

Untitled.rar

 

I can also find some interesting patterns, at around 0x155725a for example. These are probably textures.

0x155725a.thumb.png.2402207523f54604744f8d85bf3df3e2.png

 

Opening the data with Model Researcher doesn't seem to reveal anything that is too recognizable though, but I would seem to get the best results when using 'Short_sign'.

vertices.thumb.png.431fd304d348ef7b98f2f93a19913266.png

Edited by Natsuki Okusawa
Link to comment
Share on other sites

Posted (edited)

Thank you for the response. The threads about eeMemory on Xentax were for different games though. There wasn't even any mention of this particular game on Xentax. I'm also almost certain that the procedures on getting models from eeMemory depends on the game, especially considering that PS2 games tend to have different formats per game.

Edited by Natsuki Okusawa
Link to comment
Share on other sites

That model format(from Untitled.rar) is very similar to SH2/3/4 PS2 just with different file header:  0x00 header size(4 bytes), 0x04 pointer to textures(4 bytes), 0x08 pointer to textures2(4 bytes), 0x0C empty(4 bytes), 0x10 empty(16 bytes). Then at 0x20 is Main mesh header. I posted a topic about SH4 but for XBOX , Main mesh header is similar, you can check here: SH4 XBOX model

Sadly, XBOX and PS2 have different submeshes format, I never understood PS2 submeshes, those are a little bit complex(specifically submesh headers) for what I know, vertices and Uvs are shorts. I think the only person who understood PS2 submeshes was Murugo, he created a plugin for Blender so maybe you can examine his model script to find some answers: Murugo's sh2/3 model plugin

NOTE: Tokimeki Memorial 3 was released in 2001 so that means TM3 models could have more differences in Main mesh header and submeshes headers because this game is older that SH2/3/4..

Edited by roocker666
Link to comment
Share on other sites

Posted (edited)

Speaking of Silent Hill, a part of me was actually hoping that Tokimeki Memorial 3 uses a similar model format as the Silent Hill games from around the time of its release.

I was thinking of trying the Blender script for the PS2 SH2/3 models on the file (Untitled) that I provided to see if it will work, but I couldn't even get the script to work at all unfortunately, as I'm getting errors related to 'gsutil', which I think is some library necessary for the Blender script to work.

Well anyway, both Silent Hill 2 and Tokimeki Memorial 3 were released in 2001, with SH2 being released a few months prior to TM3.

Edited by Natsuki Okusawa
Link to comment
Share on other sites

Third byte of  tag(in verts, Uvs, normals) is the count. In this case 19 HEX

Now this is an individual submesh but in other submeshes it seems like there are vertex groups.

NOTE: tags of verts, Uvs and normals change depending on the vertex buffer.  I think Uvs and normlas have padding but I am not really sure..

Edited by roocker666
Link to comment
Share on other sites

10 minutes ago, roocker666 said:

OK!, I think I understand a little bit. Signature "04010001" is for vertex buffer, then we can find 3 more, these for vertices, Uvs and probably normals. SO it should be something like this:

3 More? Haha, 04010001 is contained 5180 times in the 32 MB eeMemory.bin.

Ah, you mean "the next 3 ones"?

So you say, red (ID 0x69) is for vertices?

Edited by shak-otay
Link to comment
Share on other sites

2 minutes ago, shak-otay said:

3 More? Haha, 04010001 is contained 5180 times in the 32 MB eeMemory.bin.

Ah, you mean "the next 3 ones"?

So you say, red (ID 0x69) is for vertices?

Sorry, by three more I mean 3 different more, one for vertices, other for Uvs and other for normals. SO it is like this:

04010001 --vertex buffer tag

     04001969 -- verts tag

          vertices (shorts)

     0500196D -- UVs tag

           Uvs(shorts)

     07001965 --Normals tag

           Normals(short)

 

04010001 -- next submesh

 

Don't use eeMemory.bin file, that has like 3 or 4 different models because is PCSX2 memory. Better use Untitled.rar that is only one model.

Link to comment
Share on other sites

Maybe I should think about the creation of vertices? All I get is spheres...

0x5610: 04010001 0E00 0D69

0. 5618 13 (5680)
v -0.093750 1.589844 0.593750 
v 1.746094 0.593750 -0.093750 
...

 

edit: ooh, NO! I used a stride of 8 which was wrong. Totally.

Now it looks like so (could be "something"):

 

untitled.png

Edited by shak-otay
Link to comment
Share on other sites

1 hour ago, shak-otay said:

PS2 uses auto created face indices, iirc. But it looks like shelves then,

So I used a point cloud skinner (well, looks holey):

 

 

Yeah that looks better. So probably the model is not in T pose, it needs the bones. Well, at least we got something, lol.

Link to comment
Share on other sites

First of all, thank you very much guys for taking the time and effort to take a look at this, I highly appreciate it. I'm very glad to see that this at least have gotten somewhere.

 

5 hours ago, roocker666 said:

Don't use eeMemory.bin file, that has like 3 or 4 different models because is PCSX2 memory. Better use Untitled.rar that is only one model.

I think it would be ideal to create an extractor, like some QuickBMS script to extract or automatically pull out all of the model data present within the eeMemory file? Either that or find a way to decompress those .atp files, but that would probably be much harder and I think it would be easier to just pull them out of the eeMemory file. I also just found the file that I think is the 'Untitled' file in its .atp form, which I just attached in my other thread.

 

23 hours ago, roocker666 said:

That model format(from Untitled.rar) is very similar to SH2/3/4 PS2 just with different file header:  0x00 header size(4 bytes), 0x04 pointer to textures(4 bytes), 0x08 pointer to textures2(4 bytes), 0x0C empty(4 bytes), 0x10 empty(16 bytes). Then at 0x20 is Main mesh header. I posted a topic about SH4 but for XBOX , Main mesh header is similar, you can check here: SH4 XBOX model

I wonder if it would be possible to modify the SH4 Noesis script on the thread you linked to work with TM3 models, considering the possible similarity between the formats?

Link to comment
Share on other sites

5 hours ago, Natsuki Okusawa said:

I wonder if it would be possible to modify the SH4 Noesis script on the thread you linked to work with TM3 models, considering the possible similarity between the formats?

That will be a little hard because SH4 Noesis script is for XBOX meshes, PS2 meshes have very different format, only submeshes headers are similar. Maybe  modifying Murugo's script could be better(but Murugo's script is hard to understand for me..) because SH2 PS2 meshes format are almost identical, the only difference is in Normals; SH2 Normals don't have padding, in "Untitled" it seems like Normals have padding 2. Here is an update but I still don't know if it is correct at all(I found other tag at the end) of mesh:

 

untitled_mesh_update.PNG

Edited by roocker666
Link to comment
Share on other sites

2 hours ago, roocker666 said:

That will be a little hard because SH4 Noesis script is for XBOX meshes, PS2 meshes have very different format, only submeshes headers are similar. Maybe  modifying Murugo's script could be better(but Murugo's script is hard to understand for me..) because SH2 PS2 meshes format are almost identical, the only difference is in Normals; SH2 Normals don't have padding, in "Untitled" it seems like Normals have padding 2. Here is an update but I still don't know if it is correct at all(I found other tag at the end) of mesh:

Alright, I see. Did you at least got Murugo's script to work or installed in Blender? Unfortunately for me, as I mentioned earlier, I couldn't get the script to work at all. 😢

Link to comment
Share on other sites

8 hours ago, Natsuki Okusawa said:

Alright, I see. Did you at least got Murugo's script to work or installed in Blender? Unfortunately for me, as I mentioned earlier, I couldn't get the script to work at all. 😢

Yes but it loads the skeleton only, no submeshes. Maybe because there are some differences in submeshes(and probably their headers). The skeleton works, all bones are connected.

I added SH2 header in Untitled and saved it as .mdl

 

untitled_skeleton.PNG

  • Like 2
Link to comment
Share on other sites

Posted (edited)
11 hours ago, roocker666 said:

Yes but it loads the skeleton only, no submeshes. Maybe because there are some differences in submeshes(and probably their headers). The skeleton works, all bones are connected.

I added SH2 header in Untitled and saved it as .mdl

Nice! So the Blender script can at least import the 'Untitled' file. I guess that would mean taking a look at the script can give some hints on how to get the bones?

Anyway, how did you install the script? Did it involved compiling something? When I tried to install and use the script, I keep getting errors related to 'gsutil'.

Edited by Natsuki Okusawa
Link to comment
Share on other sites

1 hour ago, Natsuki Okusawa said:

Nice! So the Blender script can at least import the 'Untitled' file...

Yes but you need to edit the file in Hex editor, just delete the header from Untitled and paste the header of any SH2 model, then save it as .mdl

SH2 header includes an ID for model and pointers to mesh, textures and palettes. Pointers to textures and palettes are useless here because textures have a complete different format but are necessary to read the file(in Murugo's plugin).

About the plugin just download the zip from releases and install it on Blender 2.93.0, I got same problem with other Blender versions so I used that version.

I guess you can check the plugin to see how skeleton works(see import_model.py), you can export the skeleton from Blender if you need it but without meshes, lol. Here is Untitled with SH2 header and already saved as .mdl

Maybe meshes don't work because Normals in Untitled have 4 extra bytes or padding, in SH2 there is no padding. Maybe you can edit how to read Normals in "import_model.py" but I am not really sure if this could work, probably there are more differences in meshes format that I did not notice yet, who knows.. 

 

Untitled_SH2_hdr.rar

Edited by roocker666
Link to comment
Share on other sites

Posted (edited)

I just downloaded and installed the version of Blender you mentioned and installed the script on it, but I'm still getting the same 'gsutil' errors when I try to import anything.

May I see the files in your 'io_sh2_sh3' folder located on 'C:\Users\*\AppData\Roaming\Blender Foundation\Blender\2.93\scripts\addons\'? Thank you.

Nevermind, I finally got the script working, I was just downloading the wrong files in the Github page.

Edited by Natsuki Okusawa
Link to comment
Share on other sites

Posted (edited)

I noticed that there is an error when I import the modified Untitled file, and importing it would sometimes crash Blender for some reason.

Quote

Python: Traceback (most recent call last):
  File "C:\Users\ADMIN\AppData\Roaming\Blender Foundation\Blender\2.93\scripts\addons\io_sh2_sh3\__init__.py", line 50, in execute
    status, msg = import_mdl.load(context, **keywords)
  File "C:\Users\ADMIN\AppData\Roaming\Blender Foundation\Blender\2.93\scripts\addons\io_sh2_sh3\import_mdl.py", line 555, in load
    parser.parse(filepath)
  File "C:\Users\ADMIN\AppData\Roaming\Blender Foundation\Blender\2.93\scripts\addons\io_sh2_sh3\import_mdl.py", line 113, in parse
    self.parse_submeshes(f, model_header.submesh_count,
  File "C:\Users\ADMIN\AppData\Roaming\Blender Foundation\Blender\2.93\scripts\addons\io_sh2_sh3\import_mdl.py", line 298, in parse_submeshes
    vt for pair in [uv[loop.vertex_index] for loop in mesh_data.loops]
  File "C:\Users\ADMIN\AppData\Roaming\Blender Foundation\Blender\2.93\scripts\addons\io_sh2_sh3\import_mdl.py", line 298, in <listcomp>
    vt for pair in [uv[loop.vertex_index] for loop in mesh_data.loops]
IndexError: list index out of range

location: <unknown location>:-1

 

I went on and asked ChatGPT about the error but I kinda doubt what it said would be really helpful. (lol)

image.png.ce8bee96ed5138f43211914fe455e983.png

Edited by Natsuki Okusawa
Link to comment
Share on other sites

23 hours ago, Natsuki Okusawa said:

I went on and asked ChatGPT about the error but I kinda doubt what it said would be really helpful. (lol)

 

That helps, you can use "print(name of vertices, Normals or Uvs variable)" in import_model.py to print vertices, normals or Uvs into the console. I did that but console shows vertices of first mesh only(it should display vertices of all meshes), no Normals nor Uvs(with SH2 models it shows all fine). SO that means the problem are Normals, like I said in SH2 Normals are like this: N1(shortX, shortY, shortZ); N2(shortX, shortY, shortZ); N3(shortX, shortY, shortZ) etc.. In Untitled are: N1(shortX, shortY, shortZ, short); N2(shortX, shortY, shortZ, short); N3(shortX, shortY, shortZ, short) etc..

There is an extra short in Normals from Untitiled, So SH2 Normals have 3 shorts and Untitled has 4 shorts, fourth short is the problem. I think import_model.py reads the extra short as the start of the next Normal and that creates  a mess. If this is the problem, you need to fix how import_model.py reads Normals but the script is hard to understand.. I can't do more man, sorry.

 

Normals.png

Edited by roocker666
Link to comment
Share on other sites

Posted (edited)

Well I admit, I can't do that much either sadly. I would also often struggle in understanding this kind of stuff, though I do kind of get some of the logic but my understanding is very limited on this. From what I can see though, you at least appear to have a better understanding on this stuff than I do.

Anyway, I've been thinking of reaching out to Murugo by sending them a DM about this for them to at least take a look at this, seeing that they are on Discord, but then looks like that would be against the rules of the Xentax Discord server from what I saw in the server's rules. Should I still go on and send Murugo a DM?

Update: I just tried to send Murugo a DM but the message couldn't be sent. 🤷‍♂️ Oh well, I guess its understandable on why they may not want to get a DM.

Edited by Natsuki Okusawa
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...