chrrox, posted Sat Jun 17, 2017 11:37 pm (24192)

---

Does quickbms support these hash functions?
https://github.com/aappleby/smhasher
Horizon zero dawn is using this one
https://github.com/aappleby/smhasher/bl ... rHash3.cpp

aluigi, posted Sun Jun 18, 2017 9:44 am (24194)

---

I think yes but configuring it may be not easy, the following is probably ok for 32bit murmur:

Code:

encryption crc "" "32 0 0 21"
get SIZE asize
log MEMORY_FILE 0 SIZE
print "%QUICKBMS_CRC|x%"

shekofte, posted Sun Jul 30, 2017 6:29 pm (25110)

---

Hello sir Luigi Auriemma
every moment contact with you is so honorable for me
after all dear Luigi i come with a crazy offer !
i hope some kind of confirmation from you ?
adding a new function or element that i called it DOSWINDOW
i want to say the DOSWINDOW is an advanced case of print command !
in fact a command that allow users to open as many dos windows they need linked to quickbms !
in other words these windows will work very similar to your MEMORY_FILEs or TEMPORARY_FILEs with the exception they are smaller and are manipulative by users for versatile purposes !
i believe this concept will increase interactivity of qickbms exponentially !
making of them are so simple and just a few functions of like printf , scanf ...
the advantage of doswindows is that they can easily arranged by operating system itself and may be they can be colorful also as you know the full screen of a doswindows can be copy pasted as text ...
...
for example user can make a window for hex view of a portion of an archive in a specific offset !
windows that list all variables used by quickbms and show their value or their content ! (debugging)
or another window for manual editing cells of an array !
as well as the process of re importing Archives can become more customizable !
i guess there is many benefit ?
even the bms script itself can opened in doswindow (colored code) pluse a separate window that contains listed bms functins that they will inserted just by pressing enter button in the code window and another window for list of compression algorithms and one for list of encryptions ...
and in the end i have a dream to write bms script while qickbms attached to the archive and by adding each line of code at the same time i see the value of the variable or when i define an offset i see the hex section of that archive offset ...
============================
DEAR Luigi Auriemma i apologize if my idea looks foolish and the verbose text cause headache .

aluigi, posted Tue Aug 08, 2017 9:18 pm (25330)

---

Many time ago I had in mind to implement some step-by-step debugging feature for showing information like the current content of the variables and the content of the input file at the current offset in some windows.
The idea was abandoned because the time and effort for implementing it is not worth the real advantages it gives.
For example for me (I mean for the personal use I make of quickbms) it's enough the -V option together with the "less" tool, never had the need of additional debugging in all these years

Acewell, posted Mon Aug 14, 2017 10:16 am (25499)

---

hi, can you add this marmoset mview decompression to quickbms?
http://forum.xentax.com/viewtopic.php?p=119352#p119352

aluigi, posted Mon Aug 14, 2017 10:43 am (25500)

---

Sure.

Acewell, posted Sat Sep 09, 2017 11:18 am (26385)

---

Acewell wrote:

hi, can you add this marmoset mview decompression to quickbms?
http://forum.xentax.com/viewtopic.php?p=119352#p119352

aluigi wrote:

Sure.

what is the name you gave this compression, i need to update my mview script.

aluigi, posted Sat Sep 09, 2017 12:12 pm (26387)

---

artstation
I didn't have samples to test it so let me know if it works correctly

Acewell, posted Sat Sep 09, 2017 3:51 pm (26390)

---

thanks
so far it worked on the samples i tried

BSM, posted Thu Nov 23, 2017 7:30 am (30225)

---

I'm new to QuickBMS, and while I have tried to search the forum for my doubt, I have found only a single reference to dictionaries (as pairs of key value, being keys unique, and where you can retrieve values given a key) as a question from chrrox in this thread: http://zenhax.com/viewtopic.php?p=3370#p3370

I tried to create a bms script to handle .obsp files to help Zeleboba as per this post: https://zenhax.com/viewtopic.php?f=9&t=6927, but the file structure includes a dictionary of 64 bits keys string values in a way that certain keys are used several times in the files content, and supposedly must be replaced by the corresponding strings. I know that such a complex file format maybe not the best point to start coding bms scripts, but I'm used to coding in several languages and I think I understand how to create bms scripts; it is just that the needed dictionary seems a handicap... And that is needed also for another file formats for the same game (The Guild 3).
How would you approach that?

I actually don't need to use QuickBMS for this, as I already coded my own tool for this file format, but I'm curious and maybe it could be handy for another file formats in future, for my own use and/or to share it.

aluigi, posted Thu Nov 23, 2017 12:12 pm (30230)

---

BSM wrote:

How would you approach that?

I wouldn't since that's not the job of quickbms
Extracting data from a binary file: ok.
Handling json, strings and other textual formats: not (or at least not easily and 100% compatible).

If I need a simple dictionary of keys and values I would just use arrays like:

Code:

putarray 0 -1 key1
putarray 1 -1 value1
putarray 0 -1 key2
putarray 1 -1 value2
...
putarray 0 -1 keyN
putarray 1 -1 valueN

And then using getarray accordingly but it's just a waste of time, searching a specific key many times will be very slow.

In the example you provided then you don't have just a list of keys and values, you have a full structure with nested elements (json) which is a completely different thing.
*edit* sorry I was referring to your example meant as json output handled as input.

You don't need to create key/value pair to spit out a json output from those obsp files, it's enough to parse them with a nested function and printing the output in json format like you did.
It's very easy to be honest, I do that many times with some archive formats

BSM, posted Thu Nov 23, 2017 12:29 pm (30236)

---

aluigi wrote:

BSM wrote:

How would you approach that?

I wouldn't since that's not the job of quickbms
Extracting data from a binary file: ok.
Handling json, strings and other textual formats: not (or at least not easily and 100% compatible).

I guessed it, but thanks anyway for the answer.

aluigi wrote:

In the example you provided then you don't have just a list of keys and values, you have a full structure with nested elements (json) which is a completely different thing.

Well, in fact the dictionary of strings contains both the filenames and the strings used inside each file, so if just unpacking to extract the files in binary form, it could be done without using the real filenames and full subfolders path, if not using the dictionary. For another file packages (.oppc), the content is a bunch of related meshes, textures and descriptors that forms a single or a multiple 3D element, so just the descriptors uses some nested properties (like json), but the different parts can be assembled into 3D objects by reading into that nested properties how all parts are related (the walls, the roof, etc, with their respective coordinates and other properties).
Also the keys are some kind of 64 bits hash (same string in different files uses the same key, so I guess it is a hash), but with an unknown algorithm; unknown for me, it is nothing I have tried, including any type of Checksum, so I am unable to pack different files, if not using same names (by using the same hashes).
Anyway, I coded a tool in C#, able to unpack and then even to translate the unpacked files into human readable text files (not polished the last part, as there are at least 2 different byte codes to produce different types of arrays, but at least to give an idea).

aluigi, posted Thu Nov 23, 2017 12:46 pm (30238)

---

With quickbms you can easily handle nested content if there are no references outside the current tree.
For example archives with a nested TOC are very easy to extract with quickbms.

episoder, posted Tue Jan 02, 2018 9:07 pm (31240)

---

lemme do a feauture request. for those sorta files i write headers for. is it possible to flag a log from a memory file (ouput header assembly) as dummy for reimporting? example...

Code:

dmlog outname 0 128 memory_file ; file header assembly
append
log outname data datasize 0

means it can write the header the regular way and extract and apped data seperate. when reimporting the dmlog header is skipped and only the data is re-inserted.

aluigi, posted Wed Jan 03, 2018 10:44 am (31248)

---

Yeah good idea.
I solved the problem by simply allowing the -. option to do the job without additional commands.
That option is used for continuing the execution of quickbms in case of errors so it fits the case since MEMORY_FILE reimporting is not possible and this is a way to "bypass" it.
This feature will be available in quickbms 0.8.2 that I will release in the next days.

aluigi, posted Wed Jan 03, 2018 7:33 pm (31261)

---

Ok guys, do you remember the idea of an alternative reimporting mode that appends the new file at the end of the archive and rewrites the fields containing offset, size and possible compressed size?
Quickbms 0.8.2 will have this new mode available as experimental by using -r twice like -r -r

It was very simple to implement this mode by adding few lines of source code and worked immediately in my tests.
That doesn't mean it will work in ALL the situations due to the obvious limitations of the various formats existent but if there are no MEMORY_FILEs involved and no operations on the offset/size/zsize fields (so "math OFFSET * 0x800" will not work), it should work correctly.

aluigi, posted Sat Jan 06, 2018 11:29 pm (31346)

---

I have found a simple way to bypass the problem of fields that are different than the original value read from the TOC, the new reimport2 mode will remember the last "math" operation performed on the variable and will reverse it for writing the correct value back in the TOC.
That means that "math OFFSET * 0x800" and "math OFFSET BASE_OFF" are perfectly supported, BUT they are not supported together which means that only one operation per variable is allowed for having a correct new rewritten field.

aluigi, posted Fri Mar 02, 2018 6:57 pm (32830)

---

quickbms 0.8.4 is almost ready since it's just a bugfix release, no new features.
Any last-minute idea is welcome

Phalasius, posted Mon Mar 05, 2018 12:48 pm (32899)

---

Hi.

I have an idea for you.

Replace "Press ENTER or close the window to quit" by a way to retry the script.

aluigi, posted Mon Mar 05, 2018 1:23 pm (32900)

---

If there was a problem the first time, how it could be solved by doing the same things a second time?

Phalasius, posted Tue Mar 06, 2018 8:30 am (32916)

---

For exemple i try many key for Sea of thieves and i need to relaunch and select 3 files/folders each time.
I think i have done 50times with no result and the process will be more faster if i just can retape a new key without relaunch.

If QuickBms understand the issue, they can propose other way ?
For Key issue => Can tape new key
File/Folder issue (block/protected...) => Can select new File/Folder

Still loop, if no folder, file, key issues find, write string to find te problem or understand the issue.

aluigi, posted Tue Mar 06, 2018 9:34 am (32917)

---

You are using unreal_tournament_4.bms in a completely wrong way.
The job of the script is extracting files with the correct key, not scanning N keys manually provided by the user (totally useless).

And no, it's not just a "loop", it's a new description with a new answer to get and starting a new process with the arguments saved when it started.
All of this because the user selected a wrong folder once in his life

Phalasius, posted Tue Mar 06, 2018 12:56 pm (32919)

---

If i don't know the key because nobody find it or share it,
And try to find it in bigExe with hexa editor, and have many result possible,
i need to test each one and i use unreal_tournament_4.bms for that..

But i understand it's the wrong way, show me the good one

aluigi, posted Tue Mar 06, 2018 3:58 pm (32923)

---

If you want to continue with that way it's enough that you use quickbms from command-line or create a file.bat, so you don't have to select script/file/folder everytime.
But it's a waste of time in any case for some reasons.
Anyway here you are OT, this is the correct topic viewtopic.php?f=9&t=1005 in which you will find also some tips.

michalss, posted Thu Mar 29, 2018 5:58 am (33641)

---

Hi luigi,

ihave an idea, make a wrapper around your functions to use them in .NET code. Like for example compressions, encryptions, coz qbms is only one on the planet which contains so many algos. This would be the best ever Thx