Search the Community

I'm currently in the early steps of a translation project for Tokimeki Memorial 3: Yakusoku No Ano Basho De. Right now I am making a tool for extracting the game's script. I have extracted the file system from the .iso. It has five .BIN files which contain all of the game's assets. At first I extracted text from Data5.BIN, as it contained uncompressed text that simply needed to be filtered from the other data. This text is almost exclusively for menus, which means the dialogue must be stored in the other .BIN files. When viewed in a hex editor, it becomes clear that the other data files (Data1.BIN - Data4.BIN) contain a large amount of compressed files with the header ATP. I know some of these contain dialog, because I can see scattered bits of Shift-JIS strings, some of which line up with dialogue I've seen in-game. Using a combination of the PCSX2 debugger and Ghidra I was able to identify where the compressed files are stored in memory when they are loaded off the disk and the function that is responsible for decompressing them. I've recreated this function in my toolset and it runs/exits without crashing but I don't think the decompression is working. It seems to have the following issues: Sometimes there are no changes between the compressed and "decompressed" ATP file (it's possible that this means the file's content was never actually compressed to begin with) Oftentimes the strings appear more jumped in the "decompressed" version than the compressed version. It doesn't stop when it needs to, running into the next ATP file and stopping halfway through "decompressing" that. Here's the assembly function taken from the PCSX2 disassembler: lbu a3,0x0(a0) addiu a0,a0,0x1 daddu t1,a1,zero addiu t0,zero,0x8 bne t0,zero,0x001E93F8 andi v0,a3,0x0001 lbu a3,0x0(a0) addiu a0,a0,0x1 addiu t0,zero,0x8 andi v0,a3,0x0001 bne v0,zero,0x001E9418 addiu t0,t0,-0x1 lbu v0,0x0(a0) addiu a0,a0,0x1 srl a3,a3,0x01 sb v0,0x0(a1) beq zero,zero,0x001E93E0 addiu a1,a1,0x1 bne t0,zero,0x001E942C srl a3,a3,0x01 lbu a3,0x0(a0) addiu a0,a0,0x1 addiu t0,zero,0x8 andi v0,a3,0x0001 beql v0,zero,0x001E9498 lbu v0,0x0(a0) addiu t0,t0,-0x1 bne t0,zero,0x001E9450 srl a3,a3,0x01 lbu a3,0x0(a0) addiu a0,a0,0x1 addiu t0,zero,0x8 andi v0,a3,0x0001 srl a3,a3,0x01 addiu t0,t0,-0x1 bne t0,zero,0x001E9470 sll a2,v0,0x01 lbu a3,0x0(a0) addiu a0,a0,0x1 addiu t0,zero,0x8 andi v0,a3,0x0001 srl a3,a3,0x01 lbu v1,0x0(a0) addiu a0,a0,0x1 addu v0,a2,v0 addiu t0,t0,-0x1 bne v1,zero,0x001E94E0 addiu a2,v0,0x2 beq zero,zero,0x001E94E0 addiu v1,zero,0x100 addiu a0,a0,0x1 lbu v1,0x0(a0) addiu a0,a0,0x1 sll v0,v0,0x08 srl a3,a3,0x01 or v1,v0,v1 beq v1,zero,0x001E9510 addiu t0,t0,-0x1 andi a2,v1,0x000F beq a2,zero,0x001E94D0 addiu a2,a2,0x2 beq zero,zero,0x001E94E0 srl v1,v1,0x04 nop lbu v0,0x0(a0) addiu a0,a0,0x1 srl v1,v1,0x04 addiu a2,v0,0x1 beq a2,zero,0x001E93E0 subu v1,a1,v1 lbu v0,0x0(v1) addiu v1,v1,0x1 addiu a2,a2,-0x1 sb v0,0x0(a1) nop bne a2,zero,0x001E94E8 addiu a1,a1,0x1 beq zero,zero,0x001E93E0 nop nop jr ra subu v0,a1,t1 and here's my attempt to recreate it for my tool set . I apologize in advance for the poor readability public static void DecodeATP(int startAddress, byte[] byteArray, String outputPath) { //Note to self: when implementing this, use integers that refer to indexes on the array as "pointers" //Use labels and gotos for the parts byte[] destination = new byte[byteArray.Length]; uint sourcePointer = (uint) startAddress + 8; //a0 in the assembly, skips 8 to avoid the ATP file header //set up a few variables to take the place of some registers in the original assembly uint v0 = 0x0; uint v1 = 0x0; uint a2 = 0x0; byte previousByte = byteArray[sourcePointer]; sourcePointer++; uint destinationPointer = 0; uint counter = 8; byte buffer; Part1: //Part1 v0 = (byte)(previousByte & 0x1); if (counter == 0) { previousByte = byteArray[sourcePointer]; sourcePointer++; counter = 8; } else { goto Part2; } Part2: //Part2 counter--; if (v0 == 0x0) { buffer = byteArray[sourcePointer]; sourcePointer++; previousByte = (byte)(previousByte >> 0x1); destination[destinationPointer] = buffer; destinationPointer++; goto Part1; } else { goto Part3; } Part3: //Part3 previousByte = (byte)(previousByte >> 0x1); if (counter == 0) { previousByte = byteArray[sourcePointer]; sourcePointer++; counter = 8; } else { goto Part4; } Part4: //Part4 v0 = (byte)(previousByte & 0x1); v0 = byteArray[sourcePointer]; if (v0 == 0x0) { goto Part7; } else { counter--; previousByte = (byte)(previousByte >> 0x1); if (counter != 0){ goto Part5; } previousByte = byteArray[sourcePointer]; sourcePointer++; counter = 8; goto Part5; } Part5: //Part5 v0 = (byte)(previousByte & 0x1); previousByte = (byte)(previousByte >> 0x1); counter--; a2 = (byte)(v0 << 0x1); if (counter != 0) { goto Part6; } else { previousByte = byteArray[sourcePointer]; sourcePointer++; counter = 8; goto Part6; } Part6: //Part6 v0 = (byte)(previousByte & 0x1); previousByte = (byte)(previousByte >> 0x1); v1 = byteArray[sourcePointer]; sourcePointer++; v0 = (byte)(a2 + v0); counter--; a2 = (byte)(v0 + 0x2); if (v1 == 0) { v1 = 0x100; } goto Part9; Part7: //Part7 sourcePointer++; v1 = byteArray[sourcePointer]; sourcePointer++; v0 = (byte)(v0 << 8); previousByte = (byte)(previousByte >> 0x1); v1 = (byte)(v0 | v1); counter--; if (v1 == 0x0) { goto Part11; } else { a2 = (byte)(v1 & 0xf); a2 = a2 + 0x2; if(a2 == 0x0) { goto Part8; } else { v1 = (v1 >> 0x4); goto Part9; } } Part8: //Part8 v0 = byteArray[sourcePointer]; sourcePointer++; v1 = (byte)(v1 >> 4); a2 = (byte)(v0 + 1); goto Part9; Part9: //Part9 v1 = (byte)(destinationPointer - v1); if (a2 == 0) { goto Part1; } else { goto Part10; } Part10: //Part10 v0 = byteArray[v1]; v1++; a2--; destination[destinationPointer] = (byte)v0; destinationPointer++; if (a2 != 0x0) { goto Part10; } else { goto Part1; } Part11: destination = SubArray(0, (int)destinationPointer, byteArray); File.WriteAllBytes(outputPath, destination); return; } Could anyone help me identify where the problem is?

Hi the post I'm going to be referring to is: The models are compressed and I don't know how to decompress them, which makes them pretty much unusable in their current state. In this post, the author has a decompressed version of the model file (albeit not a particularly useful model example, but certainly something nonetheless) I don't have a compressed vs decompressed state version of that exact model for comparison, but I will provide you here with a few compressed files and hopefully someone can figure out something with it. Thanks! https://www.dropbox.com/scl/fo/t8qwu7z9o71uwlmz8wom6/AJea2uhH3bbC20PukBSTH3U?rlkey=a88aen4vx9m2a5xhg90llvag3&st=xhtsq952&dl=0

Hello. I would like to share with you my private tutorial links collection. These are useful topics from Xentax, Zenhax and any other reverse engineering related sites that I was able to find in the Internet. I have put them into categories. You may encounter situation when one tutorial fits to few categories, but it is placed only in the category that it fits most. Also, you may encounter tutorials with "[PL]" mark, these are tuts in Polish language. If you don't understand this language, just skip them. Also be aware that I haven't read/watch some of these tutorials yet, so I can't recommend you which tutorial is the best choice, sometimes I'm still learning new things just like you and on this list you may find tuts that are not very useful for you, so don't blame me for that. 🙂 Just pick your favourite category and start learning. If you are complete newbie, I would recommend to start with tuts from category "Basic knowledge". If you have any other good source of knowledge, you can write a comment below and I will update this tutorials collection. Also remember that those links may be dead some day, so don't blame me for that and just make a copy for private use 🙂 So that's it. Enjoy. 😄 Links to tutorials: Basic knowledge / Archive extraction - Definitive Guide To Exploring File Formats - MultiExCommander Manual - What is a File Format? - Let's MultiEx - MultiEx Commander 4.5 YouTube tutorials - MexScript Documentation - Analyzing and Reverse Engineering a Game Archive - Key points to successful hex reading - Running a command-line tool on all the files of a directory and sub-folders (batch) - Overview of game file formats and archives - REWiki - Reverse engineering hints - REWiki - Links to reverse engineering resources - HOWTO-Reverse Engineering QuickBMS -QUICKBMS GUIDE - QUICKBMS GUIDE 2 ZLIB - QUICKBMS GUIDE 3 IF ELSE - Basic BMS Scripting - QuickBMS documentation - QuickBMS crc engine - QuickBMS - Reimporting files in the archives - QuickBMS - Scan all the supported compressions - Example of archive format reversing with QuickBMS (medium) - Practice with archives and quickbms scripts - [TUTORIAL] Making BMS Scripts Encryption tutorials - How to guess basic obfuscations: xor and sum/rot - Reverse simple decryption - Reversing AES Keys UE4 + AES Keys Collection - How To Get PAK RSA Key (Wolcen / CryEngine Games) - Retrieving ZIP passwords from games - the zero skills way - Retrieving ZIP passwords from games - the debugger way - Retrieving ZIP passwords from games using plain-text attack - Hacking Zip Passwords (C9) - Checksum / CRC scanning - Decrypt any vita game (no custom firmware needed) - How Unpack Themida 2.x.x or How Unpack Themida 2.x.x (WXP) - SM Hasher / Murmur3 hash + Wiki + Python implementation - LCG + sources + Cracking LCG + LCG in VC2 - Decrypt unity 3d files with Frida - [Tutorial] How to get Allegro passwords Hash tutorials - [Tutorial] How to restore hashed filenames from archives Compression tutorials - How to recognize the compression algorithms with your eyes - LZMA SDK (binaries + source code + specification) + LZMA Wiki - Bc7 and bc5 decompression and compression - Offzip - deflate/zlib scanning and extraction - ZSTD compression + documentation - Reverse Engineered old Compression Algorithm for Frogger - Oodle Data Compression + Oodle compressor + Oodle wiki page - RLE Compression explained + RLE Wiki + RLE in Python - LZ77 explained + LZ77 encoding + LZ77 decoding + LZ77 in Python - LZSS Compression + LZSS explained - LZ Compression + Open-source LZ Compressors Debugging / Decompilation / Disassembling - Lenas Reversing for Newbies - Reversing Spider-Man 2000 - Real World Decompilation with IDA Pro - IDAPython conditional breakpoints - The Beginner's Guide to IDAPython - Using IDAPython to Make Your Life Easier Part1 + Part 2 + Part 3 - Reverse Engineering with Ghidra - How to Reverse Engineer with IDA Pro Disassembler - How to reverse engineer functions - Debugging “Dust: A Tale of the Wired West” - Reverse Engineering Dust: Uncovering Game Scripts - Reverse Engineering an Xbox360 Game - Reverse engineering C programs - bin 0x10 - Memory Hacking — External Signature/Pattern Scanning Tutorial - Remote Debugging with IDA Pro (Windows) + Connecting host with VirtualBox - Remote Debugging with IDA Pro (Linux) - Basic IDA Python Scripting Hooking / DLL Injection - Simple C++ DLL Injector Source Code Tutorial - Quick and EZ - Dll Injection Explained (how it works + source code) - Manual Mapping DLL Injection Tutorial - Blizzhackers DLL Injection - API Hooking and DLL Injection on Windows - [PL]ReverseCraft #7 - Inline hooks, DLL injection - [PL]DLL INjection by Maciej Pakulski (PDF) - C/C++ Memory Hacking — Function Hooking / Detouring + How Function Hooking / Detouring Works - C++ Detour / Hooking Function Tutorial - [Tutorial] The different ways of hooking - API Hooking with MS Detours + MS Detours 4.0.1 description + source - Hooking tutorials - Inline hooking in Windows - C++ Internal Trampoline Hook Tutorial - OpenGL Hook - MinHook - The Minimalistic x86/x64 API Hooking Library + source on github - DLL Hijacking - [Tutorial] Simplest function hooking with IDA, Detours & C++ - [Tutorial] DetourFunction __thiscall - How to Hook Functions - Code Detouring Guide - Resources About Hooking - [Tutorial] Hooking Java Functions Console Tutorials - PS2 Texture Swizzling - PS1 LibCrypt tutorial + LibCrypt Sectors Mappings + LibCrypt Bible - Gears: A look Inside the Final Fantasy VII Game Engine - Convert Ps3 to Debug 4.21 - Extract Samples and MIDI files from Square PS2 Games - Extracting Xbox 360 game files - Reverse Engineering Xbox360 SaveGame (Dead Space) - Extract + Decrypt 3ds Roms - [HOW-TO] Assembling 3DS Homebrew for Gateway - [Tutorial] How to Decrypt, Extract & Rebuild 3DS|CIA|CXI Files - PS2 Hidden files + this + this + this + IsoBuster + Ratchet and Clank PS2 discoveries + Ratchet and Clank Hidden files extractor + this - PS2 Modding Tutorials + PS2 Modding Tools + PS2 AIO Project - [TUTORIAL] [PS2DEV] An Introduction to PS2DEV and it's History - Racing Lagoon Hacking Deep Dive (Translating PS1 games - tutorial) DOS Tutorials - DOS games reverse engineering guide - DOS file formats / encryptions / compressions etc. - DOS game "Nomad" (1993) documentation and tools Audio - The basics of VGM ripping + VGM Ripping Tools - Common Sound Formats - How to scan audio codecs with ffmpeg - Tutorial on vg-resource - Extract Binary Domain's Music/Sound (CRI .cpk bgm) - Batman Arkham Knight - Extract Soundtrack - Multimedia Wiki (Audio, Video, Codecs) - TXTH files documentation Graphics - Finding graphics - Xentax Wiki - Image file formats - Tutorial : How to edit EA's FIFA PS2 TEXTURE Source code + documentation - PS1 BRender Engine (Harry Potter 1, Harry Potter 2 etc.) - PS1 BuggyBunny (Bugs Bunny) - Python Tools Collection - KAO2_PAK + documentation - J2ME game Stalker - Soul Calibur 3 Reversing Books - Reverse Engineering for Beginners + Reverse Engineering challenges + Compiler Explorer - Game Hacking: Developing Autonomous Bots for Online Games - Reversing: Secrets of Reverse Engineering - The IDA Pro Book, 2nd Edition: The Unofficial Guide to the World's Most Popular Disassembler - Mastering Reverse Engineering - Practical Reverse Engineering - [PL]Gynvael Coldwind Praktyczna inżynieria wsteczna - [PL]Asembler. Sztuka programowania 3D Models - Approaches of Parsing Bone Representations - Noesis tutorial Basic Model - Video tutorials on model formats reversing - Make_obj (C source) - Extracting simple models - Analyzing and Extracting a Game Model - [Tutorial] How to extract bones from any games with Noesis - Sanae3D - 3D format converter - Background Knowledge of Bone Stuffs - Extract 3D models from binary files - Writing an Obj Convertor in C - [crash course] How to get multiple submeshes using Make_H2O - Very Basic model format conversion (Shaiya). - INTEL GPA mesh ripper tutorial + tool (x32,x64,DX9,10,11) - 3ds Ram Dump - [Tutorial] Exploring model formats: 9 Dragon - Noesis tutorial Zlib - Video tutorial on full model format reversing (Planetside) krystalgamer's Lair - Marvel Avengers - Writing a server emulator - Spider-Man 2000 - Save file protection - Spider-Man 2000 - Re-enabling a cheat code - Spider-Man 2000 - Save file content exploration - Spider-Man 2000 - PKR Format - Spider-Man 2000 - Custom File Loader - Spider-Man 2000 - Apple to the Rescue! Mobile (Android, iOS, J2ME) - OWASP guide - OWASP guide crackmes source code (Warning! Big spoilers!) - Beginners Guide to Reverse Engineering Android Apps - [Android] Decrypt unity 3d files with Frida - Mobile Game Asset Download - [PL]Reverse engineering mobile app (Niebezpiecznik) - J2ME Reverse Engineering Capture The Flag - Solving CTF Challenges Part1 and Part 2 - [PL]Capture the flag Tutorials playlist Other - The Ultimate Game Hacking Resource - Low-Level Programming University - 1024bar tutorial - [PL]ReverseCraft blog and ReverseCraft playlist - [PL]Modding tutorials - UE4 games texture replace - nullsecurity.org - crackmes.one - RCE Tools Library - GHB1 - Start Here Beginner Guide to Game Hacking - Ripping Tutorials - Searching text strings using Total Commander