[PS2] Tokimeki Memorial 3 - Getting Models from eeMemory?

[Natsuki Okusawa]

On 4/20/2024 at 5:44 PM, roocker666 said:

There is an extra short in Normals from Untitiled, So SH2 Normals have 3 shorts and Untitled has 4 shorts, fourth short is the problem. I think import_model.py reads the extra short as the start of the next Normal and that creates  a mess. If this is the problem, you need to fix how import_model.py reads Normals but the script is hard to understand.. I can't do more man, sorry.

I've been thinking about this. Would it be possible to get better results if the script were to be modified to skip the extra short 'Untitled' has?

The extra short, along with the '01010001' tag might be somehow related to the cel shading the game uses? I'm just trying to guess.

[Natsuki Okusawa]

I just went on and posted an 'issue' regarding this on the GitHub repository of Murugo's script, and they replied and modified the script to support models from this game. (Thanks, Murugo)

https://github.com/Murugo/Misc-Game-Research/issues/9#issuecomment-2084724680

I guess the next thing needed now would be either decompressing those .atp files or a model extractor script for the eeMemory file.

Edit: I'm marking this post as the 'solution', but the credit goes to Murugo. Once again, I would like say thank you to all of you again for taking the time for this. I highly appreciate all of your efforts.

Edited April 30, 2024 by Natsuki Okusawa

[roocker666]

16 hours ago, Natsuki Okusawa said:

I just went on and posted an 'issue' regarding this on the GitHub...

Nice, it was a good idea asking on Murugo's github. You can post those .atp files on "Game aAchive" maybe someone can help you there. The problem with EEmemory.bin is that you need to create a lot of save states because PCSX2 only loads a few models in memory(only those that are on screen).  You can cut models from EEmemory manually like you did with "Untitiled" just search for these bytes: "03 00 FF FF 03 00 00 00 80 00 00 00" Probably EEmemory just loads like 3 or 4 models so it will be easy. 

BTW, a few Castlevania PS2 games use that model format too but those models have more differences. In SH2/3 and TM3 the model version or revision is 03, in Castlevania model version is 06, 07 or 08 can't remember very well, I analyzed those models a few years ago.. Model version appears here; "03 00 FF FF 03 00 00 00 80 00 00 00" first 4 bytes(03 00 FF FF) is  Mesh Identifier or ID, next 4 bytes(03 00 00 00) is model version. So Castlevania games use a newer model version. Interesting, right? lol

Edited May 1, 2024 by roocker666

[Natsuki Okusawa]

8 hours ago, roocker666 said:

You can post those .atp files on "Game aAchive"

But I feel like it would be more appropriate to post it in the 'Compressed files and methods' section, but unfortunately, it appears like that section is mostly inactive, and I never got replies there so far. 😢

I also did made a thread in the 'Game Archive' section but its about the .bin files and BloodRaynare made an extractor for them, but they mentioned that someone else has to look at the .atp files.

 

8 hours ago, roocker666 said:

BTW, a few Castlevania PS2 games use that model format too but those models have more differences. In SH2/3 and TM3 the model version or revision is 03, in Castlevania model version is 06, 07 or 08 can't remember very well, I analyzed those models a few years ago.. Model version appears here; "03 00 FF FF 03 00 00 00 80 00 00 00" first 4 bytes(03 00 FF FF) is  Mesh Identifier or ID, next 4 bytes(03 00 00 00) is model version. So Castlevania games use a newer model version. Interesting, right? lol

Yeah, and I guess the same engine was used between the games but with different versions.

 

8 hours ago, roocker666 said:

You can cut models from EEmemory manually like you did with "Untitiled" just search for these bytes: "03 00 FF FF 03 00 00 00 80 00 00 00" Probably EEmemory just loads like 3 or 4 models so it will be easy.

I guess I wouldn't mind just pulling them out manually, as long as the script can import the models, though automating it would have been nice. I remember that there is a QuickBMS script that extracts model and texture data from Tekken 5 eeMemory and a Noesis script can open them.

Edited May 1, 2024 by Natsuki Okusawa

[Braveman223]

Hi there,

I’ve recently been working on a project to translate Tokimeki Memorial 3 into Chinese, and I happened to come across your post.

Following some reverse engineering and disassembly analysis, I discovered that the .atp files use a form of LZSS compression. They can be decompressed using a Python script. Here is the assembly code I located (at 1E93D0):

lbu a3,0x0(a0)
addiu a0,a0,0x1
daddu t1,a1,zero
addiu t0,zero,0x8
bne t0,zero,0x001E93F8
andi v0,a3,0x0001
lbu a3,0x0(a0)
addiu a0,a0,0x1
addiu t0,zero,0x8
andi v0,a3,0x0001
bne v0,zero,0x001E9418
addiu t0,t0,-0x1
lbu v0,0x0(a0)
addiu a0,a0,0x1
srl a3,a3,0x01
sb v0,0x0(a1)
beq zero,zero,0x001E93E0
addiu a1,a1,0x1
bne t0,zero,0x001E942C
srl a3,a3,0x01
lbu a3,0x0(a0)
addiu a0,a0,0x1
addiu t0,zero,0x8
andi v0,a3,0x0001
beql v0,zero,0x001E9498
lbu v0,0x0(a0)
addiu t0,t0,-0x1
bne t0,zero,0x001E9450
srl a3,a3,0x01
lbu a3,0x0(a0)
addiu a0,a0,0x1
addiu t0,zero,0x8
andi v0,a3,0x0001
srl a3,a3,0x01
addiu t0,t0,-0x1
bne t0,zero,0x001E9470
sll a2,v0,0x01
lbu a3,0x0(a0)
addiu a0,a0,0x1
addiu t0,zero,0x8
andi v0,a3,0x0001
srl a3,a3,0x01
lbu v1,0x0(a0)
addiu a0,a0,0x1
addu v0,a2,v0
addiu t0,t0,-0x1
bne v1,zero,0x001E94E0
addiu a2,v0,0x2
beq zero,zero,0x001E94E0
addiu v1,zero,0x100
addiu a0,a0,0x1
lbu v1,0x0(a0)
addiu a0,a0,0x1
sll v0,v0,0x08
srl a3,a3,0x01
or v1,v0,v1
beq v1,zero,0x001E9510
addiu t0,t0,-0x1
andi a2,v1,0x000F
beq a2,zero,0x001E94D0
addiu a2,a2,0x2
beq zero,zero,0x001E94E0
srl v1,v1,0x04
nop 
lbu v0,0x0(a0)
addiu a0,a0,0x1
srl v1,v1,0x04
addiu a2,v0,0x1
beq a2,zero,0x001E93E0
subu v1,a1,v1
lbu v0,0x0(v1)
addiu v1,v1,0x1
addiu a2,a2,-0x1
sb v0,0x0(a1)
nop 
bne a2,zero,0x001E94E8
addiu a1,a1,0x1
beq zero,zero,0x001E93E0
nop 
nop 
jr ra
subu v0,a1,t1

decompress.py

[Natsuki Okusawa]

13 hours ago, Braveman223 said:

Hi there,

I’ve recently been working on a project to translate Tokimeki Memorial 3 into Chinese, and I happened to come across your post.

Following some reverse engineering and disassembly analysis, I discovered that the .atp files use a form of LZSS compression. They can be decompressed using a Python script. Here is the assembly code I located (at 1E93D0):

lbu a3,0x0(a0)
addiu a0,a0,0x1
daddu t1,a1,zero
addiu t0,zero,0x8
bne t0,zero,0x001E93F8
andi v0,a3,0x0001
lbu a3,0x0(a0)
addiu a0,a0,0x1
addiu t0,zero,0x8
andi v0,a3,0x0001
bne v0,zero,0x001E9418
addiu t0,t0,-0x1
lbu v0,0x0(a0)
addiu a0,a0,0x1
srl a3,a3,0x01
sb v0,0x0(a1)
beq zero,zero,0x001E93E0
addiu a1,a1,0x1
bne t0,zero,0x001E942C
srl a3,a3,0x01
lbu a3,0x0(a0)
addiu a0,a0,0x1
addiu t0,zero,0x8
andi v0,a3,0x0001
beql v0,zero,0x001E9498
lbu v0,0x0(a0)
addiu t0,t0,-0x1
bne t0,zero,0x001E9450
srl a3,a3,0x01
lbu a3,0x0(a0)
addiu a0,a0,0x1
addiu t0,zero,0x8
andi v0,a3,0x0001
srl a3,a3,0x01
addiu t0,t0,-0x1
bne t0,zero,0x001E9470
sll a2,v0,0x01
lbu a3,0x0(a0)
addiu a0,a0,0x1
addiu t0,zero,0x8
andi v0,a3,0x0001
srl a3,a3,0x01
lbu v1,0x0(a0)
addiu a0,a0,0x1
addu v0,a2,v0
addiu t0,t0,-0x1
bne v1,zero,0x001E94E0
addiu a2,v0,0x2
beq zero,zero,0x001E94E0
addiu v1,zero,0x100
addiu a0,a0,0x1
lbu v1,0x0(a0)
addiu a0,a0,0x1
sll v0,v0,0x08
srl a3,a3,0x01
or v1,v0,v1
beq v1,zero,0x001E9510
addiu t0,t0,-0x1
andi a2,v1,0x000F
beq a2,zero,0x001E94D0
addiu a2,a2,0x2
beq zero,zero,0x001E94E0
srl v1,v1,0x04
nop 
lbu v0,0x0(a0)
addiu a0,a0,0x1
srl v1,v1,0x04
addiu a2,v0,0x1
beq a2,zero,0x001E93E0
subu v1,a1,v1
lbu v0,0x0(v1)
addiu v1,v1,0x1
addiu a2,a2,-0x1
sb v0,0x0(a1)
nop 
bne a2,zero,0x001E94E8
addiu a1,a1,0x1
beq zero,zero,0x001E93E0
nop 
nop 
jr ra
subu v0,a1,t1

decompress.py 13.54 kB · 1 download

Thank you so much for sharing information and the script for the .atp files. The script works like a charm and I can get a model from the decompressed file using Murugo's modified Blender import script.

I would like to ask you regarding the textures/image format though. What have you found so far regarding it? For now I would just use manual methods to get a character model's textures.

Edited yesterday at 05:13 PM by Natsuki Okusawa

[Braveman223]

Hello,

At the moment, I have made very little progress in extracting textures and reconstructing the 3D models.

I previously wrote an article describing the purpose of most of the game's files,

【PS2游戏 心跳回忆3 初步研究-哔哩哔哩】 https://b23.tv/6n3ezAp

and I have already located and identified nearly all of the game's text data.

The game's font data is stored directly inside the ELF executable, and the font size is 32×32 pixels.

My next objective is to analyze the texture formats and reconstruct the 3D models. This will probably take a long time, but I will continue working on it. No matter how difficult the process may be, I have no intention of stopping until these remaining formats are fully understood.

[Natsuki Okusawa]

For the 3D models, maybe you can take a look at Murugo's Blender script, that might help.

For the textures, maybe the findings I posted in an another thread might help.

https://reshax.com/topic/702-ps2-tokimeki-memorial-3-textures/

Edited 16 hours ago by Natsuki Okusawa

[Braveman223]

4 hours ago, Natsuki Okusawa said:

对于 3D 模型，或许你可以看看 Murugo的 Blender 脚本，可能会有所帮助。

至于纹理方面，或许我在另一个帖子中发布的发现会有所帮助。

https://reshax.com/topic/702-ps2-tokimeki-memorial-3-textures/

Yes, I’ve already referred to Mr. Murugo’s work on GitHub and tried learning how to use Blender, and I managed to import the model. However, the next step requires mapping the textures onto it (the textures are usually located in the latter section of the extracted .atp files) and getting the character animated. As a complete beginner starting from scratch, this is definitely a challenge for me. If it’s convenient for you, feel free to add me on Discord—my username is the same as my handle here.

[Natsuki Okusawa]

I'm going to say that the textures are a bit weird here. By looking at the textures ripped from Ninja Ripper, it seems that the textures have a particular resolution or canvas, each parts have a separate texture file, and the game would place these textures in the canvas (128 x 1024 for Yukiko's winter school uniform), probably based on where the UV mapping of a part are placed? When I assign the textures, I would combine the texture parts and I would use the texture from Ninja Ripper as reference on where they are placed. As I said in the other thread though, some portions of the textures from NinjaRipper would be missing, so I would have to get the textures manually with TiledGGD to get the full texture. (Hopefully you'll get what I mean, as I'm not really sure how to put it all in words.)

An unedited texture from Ninja Ripper:

 

Merged:

Edited 8 hours ago by Natsuki Okusawa

[Natsuki Okusawa]

This might be helpful for finding character models in the files:

00653-01785 = Chitose

01842-03093 = Emi

03150-04315 = Hotaru

04372-05505 = Kazumi

05562-06750 = Mari

06807-07392 = Masaki

07419-08618 = Rika

08677-09924 = Serika

09925-10414 = Takuo

10441-11745 = Yukiko

11808-11811 = Female Character

Character models (not including the chibi ones) are 700KB to 1MB+ in size.

For some reason though, Serika's models have duplicates, and there are two instances of the winter school uniform of most of the characters.

Edited 4 hours ago by Natsuki Okusawa