[PS2] Tenchu Wrath of Heaven

[Angel333119]

Hello everyone, I am looking for a way to figure out the compression and decompress .BIN files.

Someone told me it’s LZSS. Can anyone help me, please?

TEXTURE.zip

[hohenheim]

I don't know if it will help, but I tried to analyze some game files (specifically OPTIONS.BIN), and in them I found traces of zlib as well as gzip. 

52 Offset 1F 8B (.gz) - GZIP
32 Offset 78 01 (zlib No compression(no preset dictionary)
12 Offset 78 5E (zlib Best speed (no preset dictionary)
1 Offset 78 9C (zlib Default Compression (no preset dictionary)
6 Offset 78 DA (zlib Best Compression (no preset dictionary)
41 Offset 78 20 (zlib No Compression (with preset dictionary)
9 Offset 78 7D (zlib Best speed (with preset dictionary)
5 Offset 78 BB (zlib Default Compression (with preset dictionary)
4 Offset 78 F9 (zlib Best Compression (with preset dictionary)
 

which doesn't say anything exactly but you mentioned the use of ZLSS compression, I don't rule out that since i could be wrong because I didn't research much about compression types. I focused on understanding this game's.

On the other hand, I didn't have many results with the bms script (which could help a lot), it has some scripts that I tried to use without success. Maybe the files are compressed one way for the images, and another way for the compiled images, that would explain why there is no header. Something like LZMA that came out years before the game, and is also similar to the LZ77 algorithm according to reliable sources (wikipedia lol), but I hope it is possible to at least outline some alternatives.

[hohenheim]

On 1/11/2025 at 11:58 PM, Angel333119 said:

Hello everyone, I am looking for a way to figure out the compression and decompress .BIN files.

Someone told me it’s LZSS. Can anyone help me, please?

TEXTURE.zip 171.55 kB · 14 downloads

I've been investigating the textures of another Tenchu game a bit, but I haven't forgotten about this one. The files are very compressed, but using the cheat engine, I was able to locate the uncompressed textures in the PCSX2 memory. I don't know if it will help, but they seem to have the same structure as Tenchu Fatal Shadows. Maybe if I investigate further, I can figure out how they are decompressed in memory. I don't know if I will be able to, to be quite honest, but I extracted one from the TITLEE.BIN file. It seems to be a Japanese control guide, similar to the one in the demo.

Edited July 14 by hohenheim

[hohenheim]

By the way, here's the dump I made.

T3DUMP.zip

[morrigan]

16 hours ago, hohenheim said:

I've been investigating the textures of another Tenchu game a bit, but I haven't forgotten about this one. The files are very compressed, but using the cheat engine, I was able to locate the uncompressed textures in the PCSX2 memory. I don't know if it will help, but they seem to have the same structure as Tenchu Fatal Shadows. Maybe if I investigate further, I can figure out how they are decompressed in memory. I don't know if I will be able to, to be quite honest, but I extracted one from the TITLEE.BIN file. It seems to be a Japanese control guide, similar to the one in the demo.

Actually, you can see a large number of textures with text on them by using the texture dump feature in PCSX2. However, the game's original data is, in fact, compressed, and this is also true for the PSP and Xbox versions. Moreover, this game's compression format is quite difficult, and in all these years, no one has managed to successfully reverse-engineer it.

[Rabatini]

The ideal is to dump the uncompressed raw image file through the PCSX2 memory, and not the whole dump, so it's easier to compare the compressed file with the uncompressed one.

 

[hohenheim]

1 hour ago, morrigan said:

Actually, you can see a large number of textures with text on them by using the texture dump feature in PCSX2. However, the game's original data is, in fact, compressed, and this is also true for the PSP and Xbox versions. Moreover, this game's compression format is quite difficult, and in all these years, no one has managed to successfully reverse-engineer it.

Compression is indeed quite difficult to understand, but it's been discovered how to decompress the textures of Fatal Shadows and the PSP versions (Tenchu San and Kurenai). After some research, I discovered that Russians have already managed to translate the game and its textures, modifying the files themselves, but accessing these sites is a pain; it seems they don't accept access from other countries.

[hohenheim]

1 hour ago, Rabatini said:

The ideal is to dump the uncompressed raw image file through the PCSX2 memory, and not the whole dump, so it's easier to compare the compressed file with the uncompressed one.

 

Indeed, here are some that I managed to extract.

Uncompressed.zip

[Rabatini]

16 minutes ago, hohenheim said:

Indeed, here are some that I managed to extract.

Uncompressed.zip 275.7 kB · 0 downloads

Both files are decompressed?

[hohenheim]

17 minutes ago, Rabatini said:

Both files are decompressed?

Yes, these are the format the game use, it is essentially a tim2 texture, doesn't seems to be much different than one of them. I was able to read the texture using the Console Texture Explorer

Edited July 14 by hohenheim

[Rabatini]

2 hours ago, hohenheim said:

Yes, these are the format the game use, it is essentially a tim2 texture, doesn't seems to be much different than one of them. I was able to read the texture using the Console Texture Explorer

My notebook is broken, so I'm using a corporate notebook. I can't use a hexadecimal viewer to compare the files. It would be helpful if the compressed and decompressed files were in the same file.

[hohenheim]

4 hours ago, Rabatini said:

My notebook is broken, so I'm using a corporate notebook. I can't use a hexadecimal viewer to compare the files. It would be helpful if the compressed and decompressed files were in the same file.

Sure.

Comp-Uncomp.zip

[Angel333119]

Tenchu Fatal Shadows for PS2 and PSP doesn't use texture compression, it's a simple container.. I made a program capable of viewing, exporting, and importing textures into the containers. Now I want to learn how to extract the compression from Tenchu Wrath of Heaven so I can add support for it to the tool.

If you'd like to take a look at my tool, it's available at the following link:

https://github.com/angel333119/tenchutool/releases/tag/test

It's not complete yet, it still needs many adjustments. I want to create a tool that makes it easier for translators to modify the game for their own languages.

And...

On 7/14/2025 at 4:49 PM, hohenheim said:

Indeed, here are some that I managed to extract.

Uncompressed.zip 275.7 kB · 2 downloads

Could you explain to me how you got to this result?

[hohenheim]

On 7/16/2025 at 1:44 PM, Angel333119 said:

Could you explain to me how you got to this result?

Sure, it was actually quite simple (so to speak). In PPSSPP, I was able to identify Tenchu Time of the Assasins' texture offsets and palettes by viewing the memory on the emulator GE debugger, and on the memory visualizer, and extracted them from the game files with the help of the Console Texture Explorer, so why not do the same in PCSX2? But then I discovered that you can't dump its memory, which is possible in PPSSPP. So, I just used Cheat Engine to read the emulator's memory, dumped it, copied the files, and separated them. A maximum of four came out in good quality; the others... well, they seem incomplete; the dumping isn't perfect, but I was still able to read them with ImageHeat and the Console Texture Explorer.

I'm thinking of replacing the compressed textures with the uncompressed ones to see if the game reads them. If that works, great, but I don't have much hope for it.

Anyway, as I already mentioned, maybe the "key" to decompressing the files is in the memory too, it's just a matter of debugging, but that's a lot of work, and I don't think I would be able to do it, but who knows.

[hohenheim]

On 7/16/2025 at 8:44 AM, Angel333119 said:

Tenchu Fatal Shadows for PS2 and PSP doesn't use texture compression, it's a simple container.. I made a program capable of viewing, exporting, and importing textures into the containers. Now I want to learn how to extract the compression from Tenchu Wrath of Heaven so I can add support for it to the tool.

If you'd like to take a look at my tool, it's available at the following link:

https://github.com/angel333119/tenchutool/releases/tag/test

It's not complete yet, it still needs many adjustments. I want to create a tool that makes it easier for translators to modify the game for their own languages.

And...

Could you explain to me how you got to this result?

I tried downloading it but i think i got a false positive of trojan, maybe there's something wrong with the code or maybe is just windows being a bummer

[Angel333119]

Someone shared this C# code with me, I want to leave it here for study purposes.

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;

public static class TenchuLZSS
{
    /// <summary>
    /// Compresses data to the specific LZSS format found in the game Tenchu,
    /// using Optimal Parsing for the best compression ratio.
    /// </summary>
    /// <param name="data">The raw data to be compressed.</param>
    /// <returns>A byte array containing the compressed data.</returns>
    public static byte[] Compress(byte[] data)
    {
        int dataLen = data.Length;
        if (dataLen == 0) return new byte[0];

        // cost stores the minimum cost in bits to compress the first 'i' bytes
        var cost = new double[dataLen + 1];
        for (int i = 1; i <= dataLen; i++) cost = double.PositiveInfinity;
        cost[0] = 0;

        // choice stores the best choice for position i: (0 for literal, or (length, dist) for reference)
        var choice = new (int length, int dist)[dataLen + 1];
        choice[0] = (0, 0);

        // Bit costs per operation
        const int LITERAL_COST = 9;
        const int REFERENCE_COST = 17;
        const int MIN_MATCH_LEN = 2;
        const int MAX_MATCH_LEN = 17;

        for (int i = 1; i <= dataLen; i++)
        {
            // Option 1: Cost of treating the current byte as a literal
            cost = cost[i - 1] + LITERAL_COST;
            choice = (0, 0);

            // Option 2: Search for references (matches) that end at position i-1
            int maxSearchLen = Math.Min(MAX_MATCH_LEN, i);
            for (int length = MIN_MATCH_LEN; length <= maxSearchLen; length++)
            {
                int startOfMatch = i - length;
                var matchStr = new Span<byte>(data, startOfMatch, length);

                int windowEnd = startOfMatch;
                int windowStart = Math.Max(0, windowEnd - 4095);
                
                var window = new Span<byte>(data, windowStart, windowEnd - windowStart);
                int posInWindow = LastIndexOf(window, matchStr);

                if (posInWindow != -1)
                {
                    int dist = startOfMatch - (windowStart + posInWindow);
                    if (cost[startOfMatch] + REFERENCE_COST < cost)
                    {
                        cost = cost[startOfMatch] + REFERENCE_COST;
                        choice = (length, dist);
                    }
                }
            }
        }

        // Reconstruct the optimized data stream
        var controlBits = new List<int>();
        var literals = new List<byte>();
        var referencesWords = new List<ushort>();
        int pos = dataLen;

        while (pos > 0)
        {
            var (length, dist) = choice[pos];
            if (length == 0) // It's a literal
            {
                controlBits.Add(1);
                literals.Add(data[pos - 1]);
                pos -= 1;
            }
            else // It's a reference
            {
                controlBits.Add(0);
                ushort referenceWord = (ushort)(((dist & 0xFFF) << 4) | ((length - 2) & 0xF));
                referencesWords.Add(referenceWord);
                pos -= length;
            }
        }

        controlBits.Reverse();
        literals.Reverse();
        referencesWords.Reverse();

        var references = new List<byte>();
        foreach (ushort word in referencesWords)
        {
            references.AddRange(BitConverter.GetBytes(word));
        }

        controlBits.Add(0);
        references.AddRange(BitConverter.GetBytes((ushort)0));

        return BuildTenchuLzssStream(controlBits, literals.ToArray(), references.ToArray());
    }
    
    /// <summary>
    /// Builds the final file in the Tenchu LZSS format.
    /// </summary>
    private static byte[] BuildTenchuLzssStream(List<int> controlBits, byte[] literals, byte[] references)
    {
        using (var stream = new MemoryStream())
        using (var writer = new BinaryWriter(stream))
        {
            var controlStream = new List<byte>();
            int bitIndex = 0;
            while (bitIndex < controlBits.Count)
            {
                uint controlWord = 0;
                for (int i = 0; i < 32 && bitIndex < controlBits.Count; i++, bitIndex++)
                {
                    if (controlBits[bitIndex] == 1)
                    {
                        controlWord |= (1u << (31 - i));
                    }
                }
                controlStream.AddRange(BitConverter.GetBytes(controlWord));
            }

            int offsetLiterals = 8 + controlStream.Count;
            int offsetReferences = offsetLiterals + literals.Length;

            writer.Write(offsetLiterals);
            writer.Write(offsetReferences);
            writer.Write(controlStream.ToArray());
            writer.Write(literals);
            writer.Write(references);

            return stream.ToArray();
        }
    }

    /// <summary>
    /// Decompresses data from Tenchu's specific LZSS format.
    /// </summary>
    /// <param name="compressedData">The compressed data.</param>
    /// <returns>A byte array with the decompressed data or null in case of error.</returns>
    public static byte[]? Decompress(byte[] compressedData)
    {
        if (compressedData.Length < 12) return null;

        using (var reader = new BinaryReader(new MemoryStream(compressedData)))
        {
            int offsetLiterals = reader.ReadInt32();
            int offsetReferences = reader.ReadInt32();

            if (offsetLiterals < 8 || offsetReferences < 8 || offsetReferences < offsetLiterals ||
                offsetReferences > compressedData.Length || offsetLiterals > compressedData.Length)
            {
                return null;
            }

            reader.BaseStream.Seek(8, SeekOrigin.Begin);
            byte[] controlStream = reader.ReadBytes(offsetLiterals - 8);
            byte[] literalStream = reader.ReadBytes(offsetReferences - offsetLiterals);
            byte[] referenceStream = reader.ReadBytes((int)(compressedData.Length - offsetReferences));

            int controlPtr = 0;
            int literalPtr = 0;
            int referencePtr = 0;
            var decompressedData = new List<byte>();
            var slidingWindow = new byte[4096];
            int windowPtr = 0;
            uint controlBits = 0;
            uint bitMask = 0;

            while (true)
            {
                if (bitMask == 0)
                {
                    if (controlPtr >= controlStream.Length) break;
                    controlBits = BitConverter.ToUInt32(controlStream, controlPtr);
                    controlPtr += 4;
                    bitMask = 0x80000000;
                }

                bool isLiteral = (controlBits & bitMask) != 0;
                bitMask >>= 1;

                if (isLiteral)
                {
                    if (literalPtr >= literalStream.Length) break;
                    byte @byte = literalStream[literalPtr++];
                    decompressedData.Add(@byte);
                    slidingWindow[windowPtr] = @byte;
                    windowPtr = (windowPtr + 1) & 0xFFF;
                }
                else
                {
                    if (referencePtr + 1 >= referenceStream.Length) break;
                    ushort referenceWord = BitConverter.ToUInt16(referenceStream, referencePtr);
                    referencePtr += 2;

                    int distance = referenceWord >> 4;
                    if (distance == 0) return decompressedData.ToArray();

                    int length = (referenceWord & 0xF) + 2;
                    int copySrcPtr = (windowPtr - distance) & 0xFFF;
                    for (int i = 0; i < length; i++)
                    {
                        byte @byte = slidingWindow[copySrcPtr];
                        decompressedData.Add(@byte);
                        slidingWindow[windowPtr] = @byte;
                        windowPtr = (windowPtr + 1) & 0xFFF;
                        copySrcPtr = (copySrcPtr + 1) & 0xFFF;
                    }
                }
            }

            return decompressedData.ToArray();
        }
    }
    
    /// <summary>
    /// Finds the last occurrence of a byte sequence inside another.
    /// </summary>
    private static int LastIndexOf(Span<byte> haystack, Span<byte> needle)
    {
        if (needle.Length > haystack.Length) return -1;
        for (int i = haystack.Length - needle.Length; i >= 0; i--)
        {
            if (haystack.Slice(i, needle.Length).SequenceEqual(needle))
            {
                return i;
            }
        }
        return -1;
    }
}
 

[Karen]

Thanks for everyone's hard work, this is great! If the tool for handling the TENCHU compressed files is finished, then I can also translate Wrath of Heaven.