This topic was archived here:

https://web.archive.org/web/20230000000000fw_/https://www.zenhax.com/viewtopic.php?t=15069

tbmq008, posted Wed Mar 31, 2021 9:35 pm (63153)

---

Code:

# KINGDOM HEARTS HD 1.5   2.5 ReMIX (Microsoft Windows)
# KINGDOM HEARTS HD 2.8 Final Chapter Prologue (Microsoft Windows)

math is_hed = 0
math is_pkg = 0

get ext extension
if ext == "hed"
   math is_pkg = 1
   callfunction handle_hed_pkg 1
elif ext == "pkg"
   math is_hed = 1
   callfunction handle_hed_pkg 1
endif

startfunction handle_hed_pkg
   open FDDE "hed" is_hed
   open FDDE "pkg" is_pkg
   get hed_size asize is_hed
   xmath hed_files "hed_size / 0x20"
   for i = 0 < hed_files
      set name string ""
      getdstring hash 0x10 is_hed
      string hash_in_str b hash
      putvarchr hash_in_str 32 0
      get file_offset longlong is_hed
      get file_size1 long is_hed
      get file_size2 long is_hed
      set name string hash_in_str
      goto file_offset is_pkg
      get file_size3 long is_pkg
      get sub_entries long is_pkg
      get compressed_file_size1 long is_pkg
      get file_info3 long is_pkg
      if sub_entries == 0
         string name   "."
         math file_offset   16
         if compressed_file_size1 <= 0xfffffff0
            log name file_offset compressed_file_size1 is_pkg
         else
            log name file_offset file_size3 is_pkg
         endif
      else
         string name1 p "%s/init." name
         xmath helper "16   (sub_entries * 0x30)"
         math file_offset   helper
         if compressed_file_size1 <= 0xfffffff0
            log name1 file_offset compressed_file_size1 is_pkg
            math file_offset   compressed_file_size1
         else
            log name1 file_offset file_size3 is_pkg
            math file_offset   file_size3
         endif
         for j = 0 < sub_entries
            string name2 p "%s/%u." name j
            getdstring sbe_name 0x20 is_pkg
            get sbe1 long is_pkg
            get sbe2 long is_pkg
            get sbe3 long is_pkg
            get sbe4 long is_pkg
            if sbe4 <= 0xfffffff0
               log name2 file_offset sbe4 is_pkg
               math file_offset   sbe4
            else
               log name2 file_offset sbe3 is_pkg
               xmath sbe3_1 "sbe3 && 0x10"
               math file_offset   sbe3_1
            endif
         next j
      endif
   next i
endfunction

this script can only extract files as-is, as a consequence it cannot handle any decompression stuff whatsoever. this is why you get some garbled files.

admassacration, posted Thu Apr 01, 2021 4:39 pm (63165)

---

Hello tbmq008.
I have a project to translate the game into Portuguese-BR and I am looking for tips on how to extract the texts from the game.
Use the bms with the script above with the files "SettingMenu.hed" and "SettingMenu.pkg".
When extracting, I have the files ".sed", ".l2d" and ".dat".
Any ideas on how to extract texts from menus and dialogues?

Thanks.

tbmq008, posted Thu Apr 01, 2021 5:24 pm (63166)

---

".sed" files are simply sound data.
as for the rest i have no idea, i went in and wrote the script in "blind mode".

GrzybDev, posted Sat Apr 03, 2021 11:19 am (63194)

---

As far as i can see all locale files starts with "@CTD", then some metadata and then UTF-16 encoded strings.
Replacing strings should be easy if someone figures out that metadata before strings

I've uploaded file where you can see results of "translating" this file to blah language (it's the settings menu in launcher),

GrzybDev, posted Mon Apr 05, 2021 12:25 pm (63227)

---

Good news, i managed to somewhat reverse engineer that format and now i can successfully extract strings (at least from file i sent previously)

Here's Python code that i used to extract file to ".po" file for translation

Code:

import struct, polib

locFile = open("test.dat", "rb")

if struct.unpack('cccc', locFile.read(4)) == (b'@', b'C', b'T', b'D'):
    _, = struct.unpack('h', locFile.read(2))  # Version?
    _, = struct.unpack('i', locFile.read(4))  # Padding

    unknown, = struct.unpack("i", locFile.read(4))
    locEntries, = struct.unpack("h", locFile.read(2))
    addrBlockOffset, _, = struct.unpack("hh", locFile.read(4))
    hashBlockOffset, _, = struct.unpack("hh", locFile.read(4))
    strBlockOffset, _, = struct.unpack("hh", locFile.read(4))
    _, = struct.unpack("i", locFile.read(4))  # Padding

    print("Entries: "   str(locEntries))
    print("----------------------")
    print("Address block offset: "   str(hex(addrBlockOffset)))
    print("Hash Block Offset: "   str(hex(hashBlockOffset)))
    print("String block offset: "   str(hex(strBlockOffset)))
    print("----------------------")
    
    locFile.seek(addrBlockOffset)

    offsets = []

    for i in range(locEntries):
        locID, setID, = struct.unpack('HH', locFile.read(4))
        offset, unknown, = struct.unpack("hh", locFile.read(4))
        offsets.append(offset)

    locFile.seek(strBlockOffset)

    i = 0
    poFile = polib.POFile()

    for offset in offsets:
        locFile.seek(offset)
        lastChar = b""

        while lastChar != b"\0":
            lastChar = locFile.read(1)
            locFile.read(1)

        strLen = locFile.tell() - offset - 2
        text = u""

        if strLen != 0:
            locFile.seek(offset)

            rawStr = locFile.read(strLen)
            text  = rawStr.decode('utf-16')

        print(str(i)   " - "   text)

        poFile.append(polib.POEntry(
            msgid=text,
            msgstr="",
            occurrences=[("test", str(i))]
        ))

        i  = 1
    
    poFile.save("test.po")

Re-importing shouldn't be a problem if output file will be no-longer than original file, but as far as i can see archive itself is pretty "flexible"

Here's proof, that this scripts outputs valid .po files:

GrzybDev, posted Mon Apr 05, 2021 2:11 pm (63233)

---

Okay, i finished both exporter and reimporter for locale files. It seems to work (i can freely modify texts inside settings menu), can someone help me identifying font files? And helping me replacing/adding characters?


Exporter:

Code:

import struct, polib

locFile = open("test.dat", "rb")

if struct.unpack('cccc', locFile.read(4)) == (b'@', b'C', b'T', b'D'):
    _, = struct.unpack('h', locFile.read(2))  # Version?
    _, = struct.unpack('i', locFile.read(4))  # Padding

    unknown, = struct.unpack("i", locFile.read(4))
    locEntries, = struct.unpack("h", locFile.read(2))
    addrBlockOffset, _, = struct.unpack("hh", locFile.read(4))
    hashBlockOffset, _, = struct.unpack("hh", locFile.read(4))
    strBlockOffset, _, = struct.unpack("hh", locFile.read(4))
    _, = struct.unpack("i", locFile.read(4))  # Padding

    print("Entries: "   str(locEntries))
    print("----------------------")
    print("Address block offset: "   str(hex(addrBlockOffset)))
    print("Hash Block Offset: "   str(hex(hashBlockOffset)))
    print("String block offset: "   str(hex(strBlockOffset)))
    print("----------------------")
    
    locFile.seek(addrBlockOffset)

    offsets = []

    for i in range(locEntries):
        locID, setID, = struct.unpack('HH', locFile.read(4))
        offset, unknown, = struct.unpack("hh", locFile.read(4))
        offsets.append(offset)

    locFile.seek(strBlockOffset)

    i = 0
    poFile = polib.POFile()

    for offset in offsets:
        locFile.seek(offset)
        lastChar = b""

        while lastChar != b"\0":
            lastChar = locFile.read(1)
            locFile.read(1)

        strLen = locFile.tell() - offset - 2
        text = u""

        if strLen != 0:
            locFile.seek(offset)

            rawStr = locFile.read(strLen)
            text  = rawStr.decode('utf-16')

        poFile.append(polib.POEntry(
            msgid=text,
            msgstr=""
        ))

        i  = 1

    poFile.save("test.po")

Reimporter:

Code:

import polib, struct

origFile = open("test.dat", "rb")
newFile = open("BEB2DD13552A07B1BAC2E1FC4DCD6837.dat", "wb")

origFile.seek(0x18)  # strBlockOffset location
strBlockOffset, = struct.unpack("h", origFile.read(2))
origFile.seek(0)

translated = polib.pofile("test.po")
newFile.write(origFile.read(0x20))

offset = strBlockOffset

for string in translated:
    strLen = 0

    if string.msgstr == "" and string.msgid == "":
        strLen = len("\0".encode("utf-16-le"))
    elif string.msgstr == "":
        strLen = len(string.msgid.encode("utf-16-le"))   2
    else:
        strLen = len(string.msgstr.encode("utf-16-le"))   2

    newFile.write(origFile.read(4))
    newFile.write(struct.pack("h", offset))
    origFile.read(2)
    newFile.write(origFile.read(2))

    offset = offset   strLen

newFile.write(origFile.read(strBlockOffset - origFile.tell()))

for string in translated:
    if string.msgstr == "" and string.msgid == "":
        newFile.write("\0".encode("utf-16-le"))
    elif string.msgstr == "":
        newFile.write(string.msgid.encode("utf-16-le"))
        newFile.write("\0".encode("utf-16-le"))
    else:
        newFile.write(string.msgstr.encode("utf-16-le"))
        newFile.write("\0".encode("utf-16-le"))

origFile.seek(-4, 2)
newFile.write(origFile.read(4))
origFile.close()
newFile.close()

admassacration, posted Fri Apr 09, 2021 4:14 pm (63295)

---

GrzybDev, good afternoon.
Is this Python script used in quickbms to export and import files?
Thank you.

Dark_Ansem, posted Mon Apr 12, 2021 11:34 am (63337)

---

Nice does this allow full unpack of the directories and access to the resources?

GrzybDev, posted Mon Apr 12, 2021 8:09 pm (63345)

---

Code that i posted here was only proof of concept

I'm currently writing an tool, for unpacking and packing all archive files (and decompiling and compiling of locale files, i temporarily removed locale tool because "encrypted" archives contains format which is a little bit diffirent from not-encrypted ones, but it will be soon available)

Code: https://github.com/GrzybDev/KingdomHeartsTool

Currently this tool are able to unpack all encrypted and not encrypted archives /w compression etc., pack tool might work only on files which are originally not-encrypted and not-compressed (because i'm not saving some important data, and i'm not able to create byte-perfect archives... yet)

Anyway, if anyone really want to use that tool asap, even if pack feature is not complete here is example on how to use my tool:
Unpack archive:

Code:

python -m khtool -v -u "B:\Kingdom Hearts HD 1 5 and 2 5 ReMIX\Image\en\bbs_first.hed" -o output

Repack archive:

Code:

python -m khtool -v -p output/bbs_first.json -o output/bbs_first

Edit: Now i finished both unpack and pack tool, so you can freely repack all game archives (if you use both unpack and pack tool on the same set of data, pack tool will create byte-perfect archive)

guitutilo, posted Fri Apr 23, 2021 7:27 pm (63539)

---

I am trying to translate the menu, the menu file is Image / SettingMenu.hed I used the following method:
python -m khtool -v -u "C:\Games\Kingdom Hearts HD 1 5 and 2 5 ReMIX\Image\SettingMenu.hed" -o output\

Then:
python -m khtool -v -l "output/SettingMenu/*.ctd" -o traduzir/
I translated the file BEB2DD13552A07B1BAC2E1FC4DCD6837.po because this file is from the menu in English. I translated:

msgid "Quit Game"
to:
msgid "Sair do Jogo"

Then:
python -m khtool -v -l "traduzir/output/SettingMenu/*.po" -o final/

I copied the files from the final folder and replaced the one in the output / SettingMenu folder and executed the command:
python -m khtool -v -p output/SettingMenu.json -o output/SettingMenu

I copied the files SettingMenu.hed and SettingMenu.pkg to the Image folder, but it didn't work

GrzybDev, posted Sat Apr 24, 2021 6:22 am (63544)

---

There's small problem with offset calculating, will fix it asap

guitutilo, posted Tue Apr 27, 2021 7:57 pm (63612)

---

When you fix it, will you notify me here on the forum or will I see it for the last commit there in git?

GrzybDev, posted Thu Apr 29, 2021 6:41 am (63629)

---

The issue is now fixed, my bad because of old code

I see that you translated msgid entry...
Dude, you never translated anything, before you start READ THE MANUAL https://www.gnu.org/software/gettext/ma ... Files.html

guitutilo, posted Thu Apr 29, 2021 12:17 pm (63638)

---

Now it's working. I managed to translate some parts of the menu and it worked, thanks.

guitutilo, posted Thu Apr 29, 2021 1:26 pm (63640)

---

I did an accentual translation, and it didn't work, how could this be done or not?
The word is: Exibicao

GrzybDev, posted Thu Apr 29, 2021 1:34 pm (63642)

---

There's no problem with my tool, it's a problem because your language uses characters that this game do not understand (hence ? instead of chars)
You will need to replace already existing characters in font file, with characters you want.

guitutilo, posted Thu Apr 29, 2021 2:00 pm (63643)

---

Could you explain to me how to do this? I don't know how to change to accept Brazilian characters.

It is strange not to accept the word a, but the word o accepts

GrzybDev, posted Thu Apr 29, 2021 2:45 pm (63645)

---

It is not a question for me, i only made a tool which allows to compile and extract strings from game files
You need to do your own research on how to modify font files, because i'm not interested in graphic files (yet)

My team provided me list of characters from font, so it is possible:

guitutilo, posted Thu Apr 29, 2021 2:48 pm (63646)

---

Ok I understand, I just want to know which file to change the font?

GrzybDev, posted Thu Apr 29, 2021 2:52 pm (63647)

---

I don't know, as i stated i only care about text files. And it's obviously not one font, but at least 20 (from what i can see)

They can be anywhere, maybe they are in .i2d files, maybe .tm2

guitutilo, posted Thu Apr 29, 2021 3:20 pm (63648)

---

ok, thanks a lot for the program, i will try to find these font files

Tachi28, posted Sun May 09, 2021 7:03 pm (63866)

---

Hello, I'm new on the forums and also new to Kingdom Hearts modding, I was looking through extracted archives of the game, but I wasn't able to locate text data of KH1&2 (found and edited ones from other games without problems). Do I need to look harder or is it possible that the khtool is currently not able to extract/decrypt them correctly? Thanks