This topic was archived here:

https://web.archive.org/web/20230000000000fw_/https://www.zenhax.com/viewtopic.php?t=4797

happydance, posted Sun Aug 20, 2017 3:49 pm (25766)

---

not sure if it's a bug or something wrong on how I code it, but i'm trying to convert some Japanese text strings from UTF8 to Unicode

so I've used this code to test it out first

Code:

get TEXT string
put TEXT unicode MEMORY_FILE
get SIZE ASIZE MEMORY_FILE
log _001.txt 0 SIZE MEMORY_FILE

so I tried wrting "???" to a UTF8 text file, but when I ran the file though quickbms 8.0 it converts wrongly a???? on hex E3 00 92 01 80 00 DF 30 FC 30

when I same the same document to Unicode I got C0 30 DF 30 FC 30 in hex

so I tried using 0.76 and alo got C0 30 DF 30 FC 30 in hex

tied using plain English text UTF8 and not Japanese characters and it seems to be properly converted to Unicode.

aluigi, posted Sun Aug 20, 2017 3:56 pm (25767)

---

Try using the following instruction at the beginning of the script:
codepage 932

You can even specify the codepage with the command-line throught the option -P 932

happydance, posted Sun Aug 20, 2017 4:27 pm (25768)

---

not sure I can use codepage 932 since 932 is SHIFT-JIS Japanese, is there a code page for UTF8 Japanese?

I also tried codepage 932

it gives me garbled text with a hex of 5D 7E 80 00 5D 7E 98 6E 92 01 7C FF

aluigi, posted Sun Aug 20, 2017 10:51 pm (25773)

---

I understand. Strings, unicode and codepages are a pain.
Do you mean that the output of quickbms 0.7.x is correct instead?

happydance, posted Fri Aug 25, 2017 10:35 pm (25880)

---

yes, it displays properly on 0.76, haven't tested on other 7.x build

actually I made that script back in January I think when 0.76 is the latest and i'm trying to rip/repack text form other files I haven't touched yet of the same game and been using the updated 8.0, then encountered that problem, after hours of testing I've used the old files and to my suspired it worked properly on 7.6

aluigi, posted Fri Aug 25, 2017 10:43 pm (25881)

---

I will investigate it.
The TODO list for quickbms 0.8.1 is now very long... eh it's almost time to start working on it

aluigi, posted Sun Sep 03, 2017 9:24 pm (26207)

---

I confirm that it was a stupid bug caused by a very bad "fix" I used for a rare endless loop issue.
It will be fixed in 0.8.1 that hopefully will be released this week.