April 26, 20242 yr Author Localization On 4/20/2024 at 5:44 PM, roocker666 said: There is an extra short in Normals from Untitiled, So SH2 Normals have 3 shorts and Untitled has 4 shorts, fourth short is the problem. I think import_model.py reads the extra short as the start of the next Normal and that creates a mess. If this is the problem, you need to fix how import_model.py reads Normals but the script is hard to understand.. I can't do more man, sorry. I've been thinking about this. Would it be possible to get better results if the script were to be modified to skip the extra short 'Untitled' has? The extra short, along with the '01010001' tag might be somehow related to the cel shading the game uses? I'm just trying to guess.
April 30, 20242 yr Author Localization Solution I just went on and posted an 'issue' regarding this on the GitHub repository of Murugo's script, and they replied and modified the script to support models from this game. (Thanks, Murugo) https://github.com/Murugo/Misc-Game-Research/issues/9#issuecomment-2084724680 I guess the next thing needed now would be either decompressing those .atp files or a model extractor script for the eeMemory file. Edit: I'm marking this post as the 'solution', but the credit goes to Murugo. Once again, I would like say thank you to all of you again for taking the time for this. I highly appreciate all of your efforts. Edited April 30, 20242 yr by Natsuki Okusawa
April 30, 20242 yr Supporter 16 hours ago, Natsuki Okusawa said: I just went on and posted an 'issue' regarding this on the GitHub... Nice, it was a good idea asking on Murugo's github. You can post those .atp files on "Game aAchive" maybe someone can help you there. The problem with EEmemory.bin is that you need to create a lot of save states because PCSX2 only loads a few models in memory(only those that are on screen). You can cut models from EEmemory manually like you did with "Untitiled" just search for these bytes: "03 00 FF FF 03 00 00 00 80 00 00 00" Probably EEmemory just loads like 3 or 4 models so it will be easy. BTW, a few Castlevania PS2 games use that model format too but those models have more differences. In SH2/3 and TM3 the model version or revision is 03, in Castlevania model version is 06, 07 or 08 can't remember very well, I analyzed those models a few years ago.. Model version appears here; "03 00 FF FF 03 00 00 00 80 00 00 00" first 4 bytes(03 00 FF FF) is Mesh Identifier or ID, next 4 bytes(03 00 00 00) is model version. So Castlevania games use a newer model version. Interesting, right? lol Edited May 1, 20242 yr by roocker666
May 1, 20242 yr Author Localization 8 hours ago, roocker666 said: You can post those .atp files on "Game aAchive" But I feel like it would be more appropriate to post it in the 'Compressed files and methods' section, but unfortunately, it appears like that section is mostly inactive, and I never got replies there so far. 😢 I also did made a thread in the 'Game Archive' section but its about the .bin files and BloodRaynare made an extractor for them, but they mentioned that someone else has to look at the .atp files. 8 hours ago, roocker666 said: BTW, a few Castlevania PS2 games use that model format too but those models have more differences. In SH2/3 and TM3 the model version or revision is 03, in Castlevania model version is 06, 07 or 08 can't remember very well, I analyzed those models a few years ago.. Model version appears here; "03 00 FF FF 03 00 00 00 80 00 00 00" first 4 bytes(03 00 FF FF) is Mesh Identifier or ID, next 4 bytes(03 00 00 00) is model version. So Castlevania games use a newer model version. Interesting, right? lol Yeah, and I guess the same engine was used between the games but with different versions. 8 hours ago, roocker666 said: You can cut models from EEmemory manually like you did with "Untitiled" just search for these bytes: "03 00 FF FF 03 00 00 00 80 00 00 00" Probably EEmemory just loads like 3 or 4 models so it will be easy. I guess I wouldn't mind just pulling them out manually, as long as the script can import the models, though automating it would have been nice. I remember that there is a QuickBMS script that extracts model and texture data from Tekken 5 eeMemory and a Noesis script can open them. Edited May 1, 20242 yr by Natsuki Okusawa
June 24Jun 24 Hi there, I’ve recently been working on a project to translate Tokimeki Memorial 3 into Chinese, and I happened to come across your post. Following some reverse engineering and disassembly analysis, I discovered that the .atp files use a form of LZSS compression. They can be decompressed using a Python script. Here is the assembly code I located (at 1E93D0): lbu a3,0x0(a0) addiu a0,a0,0x1 daddu t1,a1,zero addiu t0,zero,0x8 bne t0,zero,0x001E93F8 andi v0,a3,0x0001 lbu a3,0x0(a0) addiu a0,a0,0x1 addiu t0,zero,0x8 andi v0,a3,0x0001 bne v0,zero,0x001E9418 addiu t0,t0,-0x1 lbu v0,0x0(a0) addiu a0,a0,0x1 srl a3,a3,0x01 sb v0,0x0(a1) beq zero,zero,0x001E93E0 addiu a1,a1,0x1 bne t0,zero,0x001E942C srl a3,a3,0x01 lbu a3,0x0(a0) addiu a0,a0,0x1 addiu t0,zero,0x8 andi v0,a3,0x0001 beql v0,zero,0x001E9498 lbu v0,0x0(a0) addiu t0,t0,-0x1 bne t0,zero,0x001E9450 srl a3,a3,0x01 lbu a3,0x0(a0) addiu a0,a0,0x1 addiu t0,zero,0x8 andi v0,a3,0x0001 srl a3,a3,0x01 addiu t0,t0,-0x1 bne t0,zero,0x001E9470 sll a2,v0,0x01 lbu a3,0x0(a0) addiu a0,a0,0x1 addiu t0,zero,0x8 andi v0,a3,0x0001 srl a3,a3,0x01 lbu v1,0x0(a0) addiu a0,a0,0x1 addu v0,a2,v0 addiu t0,t0,-0x1 bne v1,zero,0x001E94E0 addiu a2,v0,0x2 beq zero,zero,0x001E94E0 addiu v1,zero,0x100 addiu a0,a0,0x1 lbu v1,0x0(a0) addiu a0,a0,0x1 sll v0,v0,0x08 srl a3,a3,0x01 or v1,v0,v1 beq v1,zero,0x001E9510 addiu t0,t0,-0x1 andi a2,v1,0x000F beq a2,zero,0x001E94D0 addiu a2,a2,0x2 beq zero,zero,0x001E94E0 srl v1,v1,0x04 nop lbu v0,0x0(a0) addiu a0,a0,0x1 srl v1,v1,0x04 addiu a2,v0,0x1 beq a2,zero,0x001E93E0 subu v1,a1,v1 lbu v0,0x0(v1) addiu v1,v1,0x1 addiu a2,a2,-0x1 sb v0,0x0(a1) nop bne a2,zero,0x001E94E8 addiu a1,a1,0x1 beq zero,zero,0x001E93E0 nop nop jr ra subu v0,a1,t1 decompress.py
June 24Jun 24 Author Localization 13 hours ago, Braveman223 said: Hi there, I’ve recently been working on a project to translate Tokimeki Memorial 3 into Chinese, and I happened to come across your post. Following some reverse engineering and disassembly analysis, I discovered that the .atp files use a form of LZSS compression. They can be decompressed using a Python script. Here is the assembly code I located (at 1E93D0): lbu a3,0x0(a0) addiu a0,a0,0x1 daddu t1,a1,zero addiu t0,zero,0x8 bne t0,zero,0x001E93F8 andi v0,a3,0x0001 lbu a3,0x0(a0) addiu a0,a0,0x1 addiu t0,zero,0x8 andi v0,a3,0x0001 bne v0,zero,0x001E9418 addiu t0,t0,-0x1 lbu v0,0x0(a0) addiu a0,a0,0x1 srl a3,a3,0x01 sb v0,0x0(a1) beq zero,zero,0x001E93E0 addiu a1,a1,0x1 bne t0,zero,0x001E942C srl a3,a3,0x01 lbu a3,0x0(a0) addiu a0,a0,0x1 addiu t0,zero,0x8 andi v0,a3,0x0001 beql v0,zero,0x001E9498 lbu v0,0x0(a0) addiu t0,t0,-0x1 bne t0,zero,0x001E9450 srl a3,a3,0x01 lbu a3,0x0(a0) addiu a0,a0,0x1 addiu t0,zero,0x8 andi v0,a3,0x0001 srl a3,a3,0x01 addiu t0,t0,-0x1 bne t0,zero,0x001E9470 sll a2,v0,0x01 lbu a3,0x0(a0) addiu a0,a0,0x1 addiu t0,zero,0x8 andi v0,a3,0x0001 srl a3,a3,0x01 lbu v1,0x0(a0) addiu a0,a0,0x1 addu v0,a2,v0 addiu t0,t0,-0x1 bne v1,zero,0x001E94E0 addiu a2,v0,0x2 beq zero,zero,0x001E94E0 addiu v1,zero,0x100 addiu a0,a0,0x1 lbu v1,0x0(a0) addiu a0,a0,0x1 sll v0,v0,0x08 srl a3,a3,0x01 or v1,v0,v1 beq v1,zero,0x001E9510 addiu t0,t0,-0x1 andi a2,v1,0x000F beq a2,zero,0x001E94D0 addiu a2,a2,0x2 beq zero,zero,0x001E94E0 srl v1,v1,0x04 nop lbu v0,0x0(a0) addiu a0,a0,0x1 srl v1,v1,0x04 addiu a2,v0,0x1 beq a2,zero,0x001E93E0 subu v1,a1,v1 lbu v0,0x0(v1) addiu v1,v1,0x1 addiu a2,a2,-0x1 sb v0,0x0(a1) nop bne a2,zero,0x001E94E8 addiu a1,a1,0x1 beq zero,zero,0x001E93E0 nop nop jr ra subu v0,a1,t1 decompress.py 13.54 kB · 1 download Thank you so much for sharing information and the script for the .atp files. The script works like a charm and I can get a model from the decompressed file using Murugo's modified Blender import script. I would like to ask you regarding the textures/image format though. What have you found so far regarding it? For now I would just use manual methods to get a character model's textures. Edited June 24Jun 24 by Natsuki Okusawa
June 25Jun 25 Hello, At the moment, I have made very little progress in extracting textures and reconstructing the 3D models. I previously wrote an article describing the purpose of most of the game's files, 【PS2游戏 心跳回忆3 初步研究-哔哩哔哩】 https://b23.tv/6n3ezAp and I have already located and identified nearly all of the game's text data. The game's font data is stored directly inside the ELF executable, and the font size is 32×32 pixels. My next objective is to analyze the texture formats and reconstruct the 3D models. This will probably take a long time, but I will continue working on it. No matter how difficult the process may be, I have no intention of stopping until these remaining formats are fully understood.
June 25Jun 25 Author Localization For the 3D models, maybe you can take a look at Murugo's Blender script, that might help. For the textures, maybe the findings I posted in an another thread might help. https://reshax.com/topic/702-ps2-tokimeki-memorial-3-textures/ Edited June 25Jun 25 by Natsuki Okusawa
June 25Jun 25 4 hours ago, Natsuki Okusawa said: 对于 3D 模型，或许你可以看看 Murugo的 Blender 脚本，可能会有所帮助。 至于纹理方面，或许我在另一个帖子中发布的发现会有所帮助。 https://reshax.com/topic/702-ps2-tokimeki-memorial-3-textures/ Yes, I’ve already referred to Mr. Murugo’s work on GitHub and tried learning how to use Blender, and I managed to import the model. However, the next step requires mapping the textures onto it (the textures are usually located in the latter section of the extracted .atp files) and getting the character animated. As a complete beginner starting from scratch, this is definitely a challenge for me. If it’s convenient for you, feel free to add me on Discord—my username is the same as my handle here.
June 25Jun 25 Author Localization I'm going to say that the textures are a bit weird here. By looking at the textures ripped from Ninja Ripper, it seems that the textures have a particular resolution or canvas, each parts have a separate texture file, and the game would place these textures in the canvas (128 x 1024 for Yukiko's winter school uniform), probably based on where the UV mapping of a part are placed? When I assign the textures, I would combine the texture parts and I would use the texture from Ninja Ripper as reference on where they are placed. As I said in the other thread though, some portions of the textures from NinjaRipper would be missing, so I would have to get the textures manually with TiledGGD to get the full texture. (Hopefully you'll get what I mean, as I'm not really sure how to put it all in words.) An unedited texture from Ninja Ripper: Merged: Edited June 25Jun 25 by Natsuki Okusawa
June 25Jun 25 Author Localization This might be helpful for finding character models in the files: 00653-01785 = Chitose 01842-03093 = Emi 03150-04315 = Hotaru 04372-05505 = Kazumi 05562-06750 = Mari 06807-07392 = Masaki 07419-08618 = Rika 08677-09924 = Serika 09925-10414 = Takuo 10441-11745 = Yukiko 11808-11811 = Female Character Character models (not including the chibi ones) are 700KB to 1MB+ in size. For some reason though, Serika's models have duplicates, and there are two instances of the winter school uniform of most of the characters. Edited June 25Jun 25 by Natsuki Okusawa
July 30Jul 30 I spent some time looking into the texture format used by the PS2 version of Tokimeki Memorial 3, mainly because I wanted to extract and edit the game textures for localization.At first, the MDL format looked completely unknown. I didn't know where the textures were stored, how the image size was saved, or how the palettes were connected to the images.After analyzing the binary data step by step, I was able to figure out the basic structure.1. General MDL StructureThe MDL file is not just a simple texture container.It is roughly organized like this:MDL file ├── File header │ ├── Model section │ ├── Model data │ ├── Material information │ └── Texture reference table │ └── Image section ├── Texture data ├── Palette data └── Other image resourcesThe file header contains offsets that point to the model section and image section.The model section size is not fixed. It depends on the model itself, such as the number of vertices, materials, and textures.2. Finding the Texture IDsThe first important discovery was the texture reference table inside the model section.The game does not store textures directly in the model data. Instead, the model contains references to texture resources.The texture table can be located through:model_offset + texture_table_offsetEach texture entry is 8 bytes:+0x00 : image index +0x04 : resource IDFor example:00000000 72140000 00000000 7D140000 00000000 85140000means the texture resource IDs are:0x1472 0x147D 0x1485These IDs are later used to find the actual image resources.3. The 0x99 Resource HeaderThe biggest breakthrough was finding the meaning of:00 00 00 99At first, I thought it might be part of the image data, but it turned out to be a marker at the end of a 32-byte resource header.Each resource looks like:32-byte header + resource dataThe header structure is:Offset Meaning 0x00 Resource ID 0x04 Atlas X 0x06 Atlas Y 0x08 Width 0x0A Height 0x0C Bits per pixel 0x0E Metadata ID 0x10 Data size 0x14 Stored size 0x1C 00 00 00 994. Finding Image Width and HeightOne thing I was stuck on for a while was where the image dimensions were stored.The answer was actually simple:header + 0x08 = width header + 0x0A = heightFor example:72 14 00 00 F8 00 08 00 08 00 08 00 38 00 40 00 00 00 50 00 00 00 00 00 00 00 00 00 00 99can be decoded as:Resource ID: 0x1472 Position: X = 248 Y = 8 Size: 8 x 8 pixels Format: 8bpp indexed color Image data: 0x40 bytesThe size also matches:8 × 8 × 1 byte = 64 bytes = 0x40which confirms the interpretation.5. Texture FormatThe textures are stored as 8-bit indexed images.The image data does not contain RGB values directly.Instead:Pixel index | v Palette lookup | v RGBA colorFor example:05 06 07means:Use palette entry 5 Use palette entry 6 Use palette entry 76. Palette DataThe palette is stored separately from the image.Palette resources have a special format:width = 0 height = 0 data size = 0x400Because:256 colors × 4 bytes = 1024 bytesthe palette size is exactly:0x400 bytesEach entry contains:R G B Avalues.7. PS2 Palette OrderingOne extra step was needed because PS2 does not store the CLUT (palette) in normal order.The palette entries need to be rearranged from the PS2 CSM1 hardware layout into normal index order before the colors display correctly.Without this step, the image appears with incorrect colors.8. Image Data AlignmentThe image data also has an alignment rule.Rows are aligned to 8 pixels:row_stride = (width + 7) & ~7For example:A 100-pixel-wide image may actually store:104 pixels per rowThe extra padding bytes need to be removed during decoding.9. Final Extraction ProcessThe whole process is now:MDL file ↓ Read file header ↓ Find model section ↓ Read texture reference table ↓ Get resource IDs ↓ Find image and palette resources ↓ Read texture header ↓ Get width / height / format ↓ Decode indexed image ↓ Apply palette conversion ↓ Generate RGBA ↓ Export PNG
Create an account or sign in to comment