NullRd, posted Wed Dec 07, 2016 12:19 pm (19330)

---

strlen calculates wrong length on unicode strings

Code:

set str binary "\x74\x00\x65\x00\x73\x00\x74\x00\x00\x00"
set str unicode str
strlen slen str
print %slen%  // SCRIPT's MESSAGE: 4  // OK

set str binary "\x74\x00\x35\x04\x73\x00\x42\x04\x00\x00"
set str unicode str
strlen slen str
print %slen%  // SCRIPT's MESSAGE: 6

set str binary "\x63\x01\x17\x01\x61\x01\x65\x01\x00\x00"
set str unicode str
strlen slen str
print %slen%  // SCRIPT's MESSAGE: 8

aluigi, posted Wed Dec 07, 2016 2:51 pm (19332)

---

It's correct, if you replace %slen% with "%slen% %str%" in the print command you can see that the lenght matches.
I guess you want to specify a different codepage with the CodePage command first.

NullRd, posted Wed Dec 07, 2016 3:34 pm (19335)

---

Thanks aluigi, "codepage 1200" solves the problem

aluigi, posted Wed Dec 07, 2016 3:58 pm (19337)

---

codepage rocks

NullRd, posted Wed Dec 07, 2016 4:06 pm (19338)

---

But now main script hangs on line "put strs unicode memory_file"

If I reset codepage ("codepage 1251") before this command, script is works fine, but I don't understand this logic. When I should set and reset the codepage to get a proper result?

1.7z

aluigi, posted Wed Dec 07, 2016 4:59 pm (19339)

---

The hanging of put unicode with the default codepage is for sure a bug... I released quickbms 0.7.7 just few days ago so who knows when I will check this thing.

Using codepage 1251 before the put command worked.
You have also to use "set str unicode str" after "get str unicode".

Anyway are you sure you really want to use quickbms for string related stuff like this?

NullRd, posted Thu Dec 08, 2016 1:27 am (19345)

---

aluigi wrote:

You have also to use "set str unicode str" after "get str unicode".

Are you sure about this? In that case, the result looks completely unreadable.

aluigi wrote:

Anyway are you sure you really want to use quickbms for string related stuff like this?

Why not?
I can use some other languages, but for me, quickbms is the easiest way to extract data from a file.
It can do a lot of useful things, without boring stuff like keeping handles, checking pointers etc.

aluigi, posted Thu Dec 08, 2016 1:56 pm (19359)

---

You are right quickbms already performs the unicode->utf8 conversion when you use the Get unicode command, so forget what I said

The reason why I suggested other solutions it's because quickbms is perfect for operations like "extract file X from offset Y of size Z" but it's a bit of a pain with string-related operations.
In your case probably you may find interesting the SLog command that automatically dumps the string correctly to find with line-feed and it's even working in reimport mode.

aluigi, posted Thu Dec 08, 2016 1:59 pm (19360)

---

This is what I mean with the Slog command:

Code:

get xz1 long
get strCount long
get xz3 long
get xz4 long
get xz5 long
get xz6 long
get xz7 longlong

goto 0x120

savepos pos
xmath x "pos   0x3E"
goto x
get x short
xmath pos "pos   (x * 0x98)   0x40"
goto pos

codepage 1251

for i = 0 < strCount
  slog "output2.txt" -1 -1 unicode
next i  = 1

NullRd, posted Fri Dec 09, 2016 9:54 am (19379)

---

Thanks, aluigi!
SLog seems to be a most effective solution.

spider91, posted Tue Dec 13, 2016 7:11 pm (19487)

---

When [output_folder] in quickbms is inside the quotes and ends with slash or backslash it gives an error. It's not very convenient when you use bath files like

Code:

for /r %%i in (*.dat) do quickbms -Y "d:\script.txt" "%%i" "%%~dpi"

because "%%~dpi" gives path that ends with slash.

https://puu.sh/sOhMd/a1fbf1c1b4.png

aluigi, posted Tue Dec 13, 2016 8:59 pm (19490)

---

The final \" is probably the problem.
Try using "%%~dpi\"

spider91, posted Wed Dec 14, 2016 1:52 pm (19501)

---

Thanks, it works with two "\"

mirh, posted Mon Jan 23, 2017 5:05 pm (20373)

---

QuickBMS crashes with this testcase.

test.zip

aluigi, posted Mon Jan 23, 2017 6:45 pm (20374)

---

Eh, unfortunately not all the TotalCommander plugins can be easily supported.
When I tested this feature many years ago (far 2010) I had success with many TC plugins but those more complex relying on specific TC features/functions/callbacks can't work correctly. I guess InstallExplorer is one of them.

I'm quite surprised someone uses this feature of quickbms I never received feedback about it.

Anyway regarding InstallExplorer, it's a jurassik plugin that has been superseed by other standalone tools for the specific installers (innounp for InnoSetup is one of the main examples).

mirh, posted Mon Jan 23, 2017 7:30 pm (20375)

---

Well, let's say Installshields are totally different beasts.
But anyway, if you say WONTFIX, I'm fine.

I guess perhaps the program shouldn't *crash* though.

aluigi, posted Mon Jan 23, 2017 7:41 pm (20376)

---

Maybe I can take a look at some TC plugins for the next release but it's just for curiosity rather than for real interest (so yeah the WONTFIX tag is probably correct).
I have not checked the crash, is it in the quickbms code or in the plugin that it's looking for a missing callback?

mirh, posted Mon Jan 23, 2017 8:13 pm (20377)

---

Code:

-------------------
*EXCEPTION HANDLER*
-------------------
An error or crash occurred:

*EH* ExceptionCode      c0000005 access violation
*EH* ExceptionFlags     00000000
*EH* ExceptionAddress   001f5724
                        001F0000   00005724 InstExpl.dll
*EH* NumberParameters   00000002
*EH*                    00000000
*EH*                    3a43c64a

Nameless, posted Fri Feb 10, 2017 5:55 pm (20755)

---

When I use recursive function, position for "get" operator is not restored to its original value after execution of that function. Is this behavior intentional?

I can deal with this quirk by saving position before calling function, but I'm curious to know is this a bug or not.

aluigi, posted Fri Feb 10, 2017 6:01 pm (20756)

---

Yes that's on purpose.
Go with:

Code:

startfunction FUNC
    savepos BCK_OFF
    ...
    goto BCK_OFF
endfunction

chrrox, posted Sun Mar 05, 2017 1:45 pm (21214)

---

having trouble with oodle in quickbms.
It works fine for some chunks but fails on others.
I am thinking mabee they are using newer version?
https://github.com/powzix/kraken

00 00 EC 80 00 00 00 00 00 00 04 00 2C 7F 93 3E 6E 1C 20 23 00 00 00 00 77 C8 03 00 93 DC D7 6B - works - sample1
00 00 F0 80 00 00 00 00 00 00 04 00 3C 49 E6 C0 E5 E4 23 23 00 00 00 00 33 CF 03 00 EC 9F 78 FB - fails - sample2
00 00 F4 80 00 00 00 00 00 00 04 00 FB B5 3B 80 18 B4 27 23 00 00 00 00 57 CE 03 00 21 D0 4B 7C - crash - sample3

decompressed size is always 0x40000
compressed size is the size of the sample

oodle_decompressors.zip

oodle_decompressors.zip

aluigi, posted Sun Mar 05, 2017 2:09 pm (21217)

---

Oodle has long history of compatibility problems of its own algorithms, it's enough to check the changelog:
http://www.radgametools.com/oodlehist.htm
What I mean is that data created with a version of oodle may not be compatible with other versions (even newer versions).
Quickbms 0.7.7 uses oodle 2.3.0.

Anyway quickbms simply calls OodleLZ_Decompress on the data "as is", it performs absolutely no operation on the bytes of the input data (that happens only if you specify a raw algorithm, not the default behaviour).

chrrox, posted Sun Mar 05, 2017 2:15 pm (21219)

---

the chunks are from the same archive.
quickbms decompressed 19gb from the file fine then failed on everything for the remaining 12gb starting at this sample here.
this is ps4 sample.
looks like it might be the 2.40 version they used.
Oodle 2.4.0 is up--New Hydra automatically selects Kraken/Mermaid/Selkie and new Mermaid compressor with slightly higher compression!

aluigi, posted Sun Mar 05, 2017 4:12 pm (21221)

---

Unfortunately 2.3.0 was the only oodle dll available on Warframe when I released quickbms 0.7.7
Currently the 2.4.1 is available but I don't know when I'm going to work on the next quickbms.
In your case it's probably more easy to write a simple tool from scratch, after all it's just one API to call... ok it's not that simple to be honest because in the past the oodle developers changed even the prototype of the main API! :O

Nameless, posted Wed Mar 08, 2017 4:23 am (21280)

---

Just thought about something... Few times I had problems with break operator, those problems often was solvable, but sometimes they give real headaches, so I have idea about solution for this problem. Why not add labels for the cycles? And when we need to break some cycle then we could point break to the label of that cycle.

Also I think that labels for the script in general are also needed, this could help a lot in writing custom decompress functions, because disassembled code often can have lots of branches and it is not that easy to unwrap such code, there also can be some platform specific optimizations inside that code which gives even more confusion. So labels are must have in such situations.