Engineers shak-otay Posted September 28 Engineers Posted September 28 (edited) I've logged referencePtr and length (==2, mostly). Decompressing missionj.bin stops very early but for k2tx_01 it doesn't seem to reach the file end, too (but seems to contain a texture): missionj.bin, ... 26462 2, 26464 2, 26466 2, 26468 2, 26470 2, 26472 2, 26474 2, 26476 2, 26478 2, 26480 2, 26482 2, 26484 2, 26486 2, 26488 9, 26490 2, 26492 2, 26494 2, 26496 2, 26498 2, 26500 2, 26502 2, 26504 2, 26506 13, 26508 17, 26510 3, dist 0 Successfully decompressed to 'mj-decpr.bin'. Compressed size: 666656 bytes Decompressed size: 66688 bytes -------------- K2tx_01, ...130712 2, 130714 2, 130716 3, 130718 3, 130720 2, 130722 2, 130724 2, 130726 2, 130728 4, 130730 4, 130732 8, 130734 16, 130736 17, 130738 17, 130740 17, 130742 17, 130744 17, 130746 17, 130748 17, 130750 17, 130752 9, 1307542, 130756 3, 130758 2, 130760 5, dist 0 Successfully decompressed to 'k2tx_01 decpr.bin'. Compressed size: 209189 bytes Decompressed size: 288000 bytes K2tx_01, decompressed: Edited September 28 by shak-otay
Members morrigan Posted October 5 Members Posted October 5 On 9/28/2025 at 7:36 PM, shak-otay said: I've logged referencePtr and length (==2, mostly). Decompressing missionj.bin stops very early but for k2tx_01 it doesn't seem to reach the file end, too (but seems to contain a texture): missionj.bin, ... 26462 2, 26464 2, 26466 2, 26468 2, 26470 2, 26472 2, 26474 2, 26476 2, 26478 2, 26480 2, 26482 2, 26484 2, 26486 2, 26488 9, 26490 2, 26492 2, 26494 2, 26496 2, 26498 2, 26500 2, 26502 2, 26504 2, 26506 13, 26508 17, 26510 3, dist 0 Successfully decompressed to 'mj-decpr.bin'. Compressed size: 666656 bytes Decompressed size: 66688 bytes -------------- K2tx_01, ...130712 2, 130714 2, 130716 3, 130718 3, 130720 2, 130722 2, 130724 2, 130726 2, 130728 4, 130730 4, 130732 8, 130734 16, 130736 17, 130738 17, 130740 17, 130742 17, 130744 17, 130746 17, 130748 17, 130750 17, 130752 9, 1307542, 130756 3, 130758 2, 130760 5, dist 0 Successfully decompressed to 'k2tx_01 decpr.bin'. Compressed size: 209189 bytes Decompressed size: 288000 bytes K2tx_01, decompressed: It seems this format is indeed troublesome.
Engineers Rabatini Posted Saturday at 05:03 PM Engineers Posted Saturday at 05:03 PM On 9/28/2025 at 7:26 AM, shak-otay said: Thank you! I got: tenchu3 decompress titlej.bin xx.bin Decompressing 'titlej.bin'... Successfully decompressed to 'xx.bin'. Compressed size: 1148592 bytes Decompressed size: 287872 bytes tenchu3 decompress texture.bin xx2.bin Decompressing 'texture.bin'... Successfully decompressed to 'xx2.bin'. Compressed size: 247456 bytes Decompressed size: 65632 bytes so for these two samples the decompressed sizes are smaller than the compressed ones (need to check the code in detail). edit: the code is ok. I compressed the decompressed K2tx_01.bin (decompressed files are here) and decompressed it. Worked like a charm. Compressed size: 209189 kB Decompressed size: 288000 kB Hello, I don´t get it. the original files of the game the decompressor do not work as should. If you take the image decompressed already and compress e then decompress, the code will work, because the code is ok for it, but not to the original compressed file.
Engineers shak-otay Posted Saturday at 07:09 PM Engineers Posted Saturday at 07:09 PM I didn't check this any further. Maybe knowing the compression ratio (factor) could help?
Engineers Solution Rabatini Posted Sunday at 08:48 AM Engineers Solution Posted Sunday at 08:48 AM (edited) 13 hours ago, shak-otay said: I didn't check this any further. Maybe knowing the compression ratio (factor) could help? Actually the LZSS provide above, is wrong, for the files. I did the reverse enginner of the algorithim, Try the tool, see if the image get right TenchuWoH_DeCompressor.zip Edited Sunday at 08:55 AM by Rabatini 3
Members morrigan Posted Sunday at 01:33 PM Members Posted Sunday at 01:33 PM (edited) 4 hours ago, Rabatini said: Actually the LZSS provide above, is wrong, for the files. I did the reverse enginner of the algorithim, Try the tool, see if the image get right TenchuWoH_DeCompressor.zip 32.72 kB · 0 downloads Excellent work! I tested it on the JP version and successfully extracted the contents. It's about the TITLEJ.BIN file mentioned in a previous post. All the K2Tx textures inside were extracted, and the image format itself is very simple. Edited Sunday at 01:36 PM by morrigan
Angel333119 Posted 12 hours ago Author Posted 12 hours ago Rabatini, thank you so much, man. Would you be interested in making the source code available? I’d really like to implement it in my tool… I’m still learning programming and this was a challenge I couldn’t handle. If you can, I’d really appreciate your help to implement it in my tool. If you don’t want to share the code, I’ll understand…
Engineers Rabatini Posted 10 hours ago Engineers Posted 10 hours ago 2 hours ago, Angel333119 said: Rabatini, thank you so much, man. Would you be interested in making the source code available? I’d really like to implement it in my tool… I’m still learning programming and this was a challenge I couldn’t handle. If you can, I’d really appreciate your help to implement it in my tool. If you don’t want to share the code, I’ll understand… Sure, I'll send you the source code; it's in C++. I don't know what language your tool is in. I'm making the compressor. If I compress it without altering anything, the game accepts it, but when I alter something, the image gets smaller. I think I'm doing something wrong. If you could edit the initial screen so I can test the compressor, I'd appreciate it. TITLEE.BIN is the last file I decompressed. I tried using Mumm-Ra's tool, but it inserts [the data] and makes the file smaller when I compress, or maybe it's Paint. I don't have much skill with graphic editing.
Members morrigan Posted 9 hours ago Members Posted 9 hours ago I've made an LZSS compressor using Golang, which is significantly faster than Python 🤣what takes Python several minutes only takes about one second with Go. However, I haven't been able to achieve the same compression ratio as the original compressor yet. My results are typically 1-6% larger than the original files, though I've tested and confirmed that the game can decompress them correctly. The insufficient compression ratio is critical because I discovered that the game crashes if the BIN file is even 1KB larger than the original. Initially, I thought there might be some kind of checksum validation, but after testing with smaller compressed chunks, the game can read and display them perfectly. This suggests the game likely allocates a fixed memory size for each BIN container. Additionally, the information for each compressed chunk within the BIN is hardcoded in the ELF file.(I tested JP ver SLPS_252.34 ) Each chunk has a 16-byte entry containing: Offset in BIN (4 bytes) Compressed size (4 bytes) Decompressed size (4 bytes) Padding (4 bytes, usually 0x00000000) For example, the TITLEJ.BIN（6 chunks） structure in the ELF (at offset 0x347520): Entry Block BIN Offset Compressed Decompressed Padding Size Size ----- ----- ---------- ----------- ------------ -------- 0 6 0x00112160 25,936 66,688 0x00000000 1 1 0x00000000 266,096 287,872 0x00000000 2 4 0x000E6C30 102,336 263,296 0x00000000 3 5 0x000FFBF0 75,120 263,296 0x00000000 4 2 0x00040F70 323,456 328,832 0x00000000 5 3 0x0008FEF0 355,648 328,832 0x00000000 Therefore, rebuilding the BIN requires either: Full rebuild: Rewrite the ELF information with new offsets and sizes Safe mode: Insert the new compressed chunk into the original BIN and pad with dummy data where needed Of course, the safe mode requires that your new compressed chunk must not be larger than the original. If anyone can achieve better compression ratios, that would be incredibly helpful!😁 compressor.zip
Members morrigan Posted 8 hours ago Members Posted 8 hours ago Thanks to Rabatini's excellent work, the door to this game has been opened wide. I've also created a texture conversion tool. the K2 Game's K2TX format: 0x00 (4 bytes, char[4]): "K2Tx". 0x04 (4 bytes, uint32): offset of pixel data. 0x08 (4 bytes, uint32): offset of palette data. 0x0C (4 bytes, uint32): size of this K2TX file 0x10 (2 bytes, uint16): width. 0x12 (2 bytes, uint16): height. 0x1C : 8bpp Pixel Swizzle Flag:???0x02 enable 0x1E : BPP Flag: 0x01 indicates 4bpp mode?? k2tx2png.py
Angel333119 Posted 8 hours ago Author Posted 8 hours ago (edited) If you want, you can use my tool — it already works with any texture container from Tenchu: Fatal Shadows. I plan to implement the containers from Wrath of Heaven. https://github.com/angel333119/tenchutool EDIT: Rabatini, my tool is written in C#. If you want, feel free to use it — it's specifically designed for the Tenchu (K2Tx) format, and it works with all uncompressed files. Give it a try and let me know what results you get. I’ll run some tests later; right now I’m at work. For anyone interested: My tool extracts and reinserts textures inside containers, and it works perfectly with the uncompressed versions for both PS2 and PSP as well. Edited 8 hours ago by Angel333119
Engineers Rabatini Posted 8 hours ago Engineers Posted 8 hours ago (edited) 1 hour ago, morrigan said: I've made an LZSS compressor using Golang, which is significantly faster than Python 🤣what takes Python several minutes only takes about one second with Go. However, I haven't been able to achieve the same compression ratio as the original compressor yet. My results are typically 1-6% larger than the original files, though I've tested and confirmed that the game can decompress them correctly. The insufficient compression ratio is critical because I discovered that the game crashes if the BIN file is even 1KB larger than the original. Initially, I thought there might be some kind of checksum validation, but after testing with smaller compressed chunks, the game can read and display them perfectly. This suggests the game likely allocates a fixed memory size for each BIN container. Additionally, the information for each compressed chunk within the BIN is hardcoded in the ELF file.(I tested JP ver SLPS_252.34 ) Each chunk has a 16-byte entry containing: Offset in BIN (4 bytes) Compressed size (4 bytes) Decompressed size (4 bytes) Padding (4 bytes, usually 0x00000000) For example, the TITLEJ.BIN（6 chunks） structure in the ELF (at offset 0x347520): Entry Block BIN Offset Compressed Decompressed Padding Size Size ----- ----- ---------- ----------- ------------ -------- 0 6 0x00112160 25,936 66,688 0x00000000 1 1 0x00000000 266,096 287,872 0x00000000 2 4 0x000E6C30 102,336 263,296 0x00000000 3 5 0x000FFBF0 75,120 263,296 0x00000000 4 2 0x00040F70 323,456 328,832 0x00000000 5 3 0x0008FEF0 355,648 328,832 0x00000000 Therefore, rebuilding the BIN requires either: Full rebuild: Rewrite the ELF information with new offsets and sizes Safe mode: Insert the new compressed chunk into the original BIN and pad with dummy data where needed Of course, the safe mode requires that your new compressed chunk must not be larger than the original. If anyone can achieve better compression ratios, that would be incredibly helpful!😁 compressor.zip 2.71 kB · 0 downloads My compressor did i better compression rate. the file reduce like 5 kb from original. when i compressed every file. like i said, the games work, if i decompress and compress with my compressor, but if i edit something, the game works, but the image do not appears, maybe i am doing something wrong in the edit graphic,. Good know about the elf file, i can implement that later. I will send here. Edited 8 hours ago by Rabatini
Engineers Rabatini Posted 7 hours ago Engineers Posted 7 hours ago (edited) When i get home, i will compile the decompressor/compressor unpack and pck tool, is one all tool. std::vector<uint8_t> compressLZSSBlock(const std::vector<uint8_t>& input) { const int MIN_MATCH = 3; // comprimento mínimo para virar par const int MAX_MATCH = 17; // (0xF + 2) const int DICT_SIZE = 4096; const size_t n = input.size(); // Dicionário igual ao do descompressor std::vector<uint8_t> dict_buf(DICT_SIZE, 0); size_t dict_index = 1; // mesmo índice inicial do descompressor size_t producedBytes = 0; // quantos bytes já foram "gerados" (saída lógica) std::vector<uint32_t> flagWords; uint32_t curFlag = 0; int bitsUsed = 0; auto pushFlagBit = [&](bool isLiteral) { if (bitsUsed == 32) { flagWords.push_back(curFlag); curFlag = 0; bitsUsed = 0; } if (isLiteral) { // bit 1 = literal (mesmo significado do descompressor) curFlag |= (1u << (31 - bitsUsed)); } ++bitsUsed; }; std::vector<uint8_t> literals; std::vector<uint8_t> pairs; literals.reserve(n); pairs.reserve(n / 2 + 16); size_t pos = 0; while (pos < n) { size_t bestLen = 0; uint16_t bestOffset = 0; if (producedBytes > 0) { // tamanho máximo possível para este match (não pode passar do fim do input) const size_t maxMatchGlobal = std::min(static_cast<size_t>(MAX_MATCH), n - pos); // percorre todos os offsets possíveis do dicionário for (int off = 1; off < DICT_SIZE; ++off) { if (dict_buf[off] != input[pos]) continue; // --- SIMULAÇÃO DINÂMICA DO DESCOMPRESSOR PARA ESTE OFFSET --- uint8_t candidateBytes[MAX_MATCH]; size_t candidateLen = 0; for (size_t l = 0; l < maxMatchGlobal; ++l) { const int src_index = (off + static_cast<int>(l)) & 0x0FFF; // valor em src_index, levando em conta que o próprio bloco // pode sobrescrever posições do dicionário (overlap) uint8_t b = dict_buf[src_index]; // Se src_index for igual a algum índice de escrita deste MESMO par // (dict_index + j), usamos o byte já "gerado" candidateBytes[j] for (size_t j = 0; j < l; ++j) { const int dest_index = (static_cast<int>(dict_index) + static_cast<int>(j)) & 0x0FFF; if (dest_index == src_index) { b = candidateBytes[j]; break; } } if (b != input[pos + l]) { // não bate com o input, para por aqui break; } candidateBytes[l] = b; ++candidateLen; } if (candidateLen >= static_cast<size_t>(MIN_MATCH) && candidateLen > bestLen) { bestLen = candidateLen; bestOffset = static_cast<uint16_t>(off); if (bestLen == static_cast<size_t>(MAX_MATCH)) break; // não tem como melhorar } } } if (bestLen >= static_cast<size_t>(MIN_MATCH)) { // --- CODIFICA COMO PAR (offset, length) --- pushFlagBit(false); // 0 = par uint16_t lengthField = static_cast<uint16_t>(bestLen - 2); // 1..15 uint16_t pairVal = static_cast<uint16_t>((bestOffset << 4) | (lengthField & 0x0F)); pairs.push_back(static_cast<uint8_t>(pairVal & 0xFF)); pairs.push_back(static_cast<uint8_t>((pairVal >> 8) & 0xFF)); // Atualiza o dicionário exatamente como o DESCOMPRESSOR: // for (i = 0; i < length; ++i) { // b = dict[(offset + i) & 0xFFF]; // out.push_back(b); // dict[dict_index] = b; // dict_index = (dict_index + 1) & 0xFFF; // } for (size_t i = 0; i < bestLen; ++i) { int src_index = (bestOffset + static_cast<uint16_t>(i)) & 0x0FFF; uint8_t b = dict_buf[src_index]; dict_buf[dict_index] = b; dict_index = (dict_index + 1) & 0x0FFF; } pos += bestLen; producedBytes += bestLen; } else { // --- LITERAL SIMPLES --- pushFlagBit(true); // 1 = literal uint8_t literal = input[pos]; literals.push_back(literal); dict_buf[dict_index] = literal; dict_index = (dict_index + 1) & 0x0FFF; ++pos; ++producedBytes; } } // Par terminador (offset == 0) pushFlagBit(false); pairs.push_back(0); pairs.push_back(0); // Flush do último flagWord if (bitsUsed > 0) { flagWords.push_back(curFlag); } // Monta o bloco final: [u32 off_literals][u32 off_pairs][flags...][literais...][pares...] const size_t off_literals = 8 + flagWords.size() * 4; const size_t off_pairs = off_literals + literals.size(); const size_t totalSize = off_pairs + pairs.size(); std::vector<uint8_t> block(totalSize); auto write_u32_le = [&](size_t pos, uint32_t v) { block[pos + 0] = static_cast<uint8_t>(v & 0xFF); block[pos + 1] = static_cast<uint8_t>((v >> 8) & 0xFF); block[pos + 2] = static_cast<uint8_t>((v >> 16) & 0xFF); block[pos + 3] = static_cast<uint8_t>((v >> 24) & 0xFF); }; write_u32_le(0, static_cast<uint32_t>(off_literals)); write_u32_le(4, static_cast<uint32_t>(off_pairs)); size_t p = 8; for (uint32_t w : flagWords) { block[p + 0] = static_cast<uint8_t>(w & 0xFF); block[p + 1] = static_cast<uint8_t>((w >> 8) & 0xFF); block[p + 2] = static_cast<uint8_t>((w >> 16) & 0xFF); block[p + 3] = static_cast<uint8_t>((w >> 24) & 0xFF); p += 4; } std::copy(literals.begin(), literals.end(), block.begin() + off_literals); std::copy(pairs.begin(), pairs.end(), block.begin() + off_pairs); return block; } @morrigan my compressor, try it, and let me know the results. Edited 7 hours ago by Rabatini 1
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now