aluigi, posted Fri Nov 13, 2015 3:52 pm (9670)

---

In the week-end I will release QuickBMS 0.7.1.
Please let me know if you have any idea or feedback regarding something to fix/improve before this new version.

aluigi, posted Fri Nov 13, 2015 8:39 pm (9681)

---

One of the things I have to improve for sure is the output to the console because the Windows console is terribly slow.
If you take a script that generates lot of output (for example an archive with many files), try to run it on Windows and Linux to see the difference in times.
I guess the only solution will be to write a new console for the GUI mode because all the tests made on SetConsole*, setvbuf and so on are failing.

aluigi, posted Mon Nov 16, 2015 2:10 pm (9787)

---

Feel free to provide any feedback about the ISO feature of quickbms 0.7.1 (just released).

aluigi, posted Tue Nov 17, 2015 12:02 am (9817)

---

The following are the features I'm evaluating for the next versions of quickbms:

• creation of a ZIP archive, just like the ISO feature. this is simple to implement and will be done in the next release.
• a configurable LZW engine like the one available in Camoto where you can set all the parameters
• a real GUI or something similar... mah
• a browser feature to view the files without extraction, there are some technical limitations I'm trying to solve: opening the selected file with the external viewers/editors requires to write the file on disk and requires to create a full GUI (directory tree visualization and options), using the memory requires to implement all the codecs and viewers inside quickbms and this will be NEVER done, a solution like a sort of virtual disk would be perfect but not possible in usermode without drivers to handle the IO. I'm thinking about something similar.

As usual, any idea, suggestion and feedback is welcome.

*edit* regarding the last point I was thinking to webdav but it writes the "remote" file to disk making it useless.

einstein95, posted Tue Nov 17, 2015 4:39 am (9821)

---

Regarding that 2nd point, I was just going to ask if you'd include the LZW compressions used in Ultima 4 and Ultima 6, U4's uses a hash dictionary (decompressor: http://puu.sh/lor3r/910bd7d61a.zip, compressor: http://puu.sh/lor4N/880c791c94.zip) but I'm not sure what's different about U6's (decompressor: http://puu.sh/lor6p/49a0aa0f12.zip, compressor: http://puu.sh/lor6X/d70ce7dde8.zip)

aluigi, posted Tue Nov 17, 2015 12:01 pm (9831)

---

Ok, I will check them when I will implement the engine.

Another point I would like to add is the possibility of distributing a quickbms.exe with an embedded script in it.
The idea came from a request of an user and by some modkits I have seen around and I think that this feature would make their life and the one of the final users a lot easier.
Maybe a sort of feature activated if you add a certain resource or append the script to quickbms.exe.
It's easy to do so it costs nothing to me.

aluigi, posted Sat Dec 05, 2015 7:16 pm (10325)

---

I have just finished to add the output to ZIP that works like the ISO -i option but it creates a ZIP file.
There are also some new decompression algorithms:
- IBM TERSE (PACK and SPACK)
- PKWARE reduce (factor 1 -> 4)

As far as I know don't exist open source versions of these algorithms so it's something (hopefully) unique

aluigi, posted Sun Dec 06, 2015 1:23 pm (10344)

---

I have just finished to make the lzw engine of camoto/libgamearchive compatible with gcc (it doesn't like std::function).
The bad news is that the compressor is a dummy so it can't be used in reimport mode:
http://www.shikadi.net/camoto/doc/libga ... ml#details

In my tests I have some problems with the data generated by the dummy compressor but it may be caused by my modifications (bitstream replacement).
Currently I don't have an alternative ready to use but the code does its job in decompression so it's ok for me.
Maybe next time I will write my C library from scratch compatible with the initializers used in filter-lzw.cpp.

I have only to fix a small thing related to how are handled the last bytes in the stream (LZW_LEFTOVER_BYTES) and then it's ready.

aluigi, posted Mon Dec 07, 2015 11:49 pm (10379)

---

@einstein95
regarding U4 and U6, I guess that the difference is just in U4 that uses a fixed size codeword.
LZW has a codeword that starts from 9 bits and can reach a size of 16 while in U4 it starts as 12 and doesn't get modified.

I guess that U6 is just a classical LZW, anyway it will be available in quickbms 0.7.2 with its own comtype.

aluigi, posted Tue Dec 08, 2015 1:11 am (10381)

---

Regarding the feature for the embedded script, I'm trying to find a solution that will work in any case and without forcing quickbms to parse its own executable.
I was thinking about the PE Checksum field but it requires to read the whole exe (quickbms is over 12Mb) and UPX sets it to zero making it useless.
Parsing the PE headers to check if the sum of all the sections match, may not work ever and it's still related to parse a part of the executable.
I was thinking about checking a constant string in the executable but it requires the modder to hex edit the executable.
Probably this feature will not be implemented, after all it's not necessary and will be used probably only in very few cases.

aluigi, posted Tue Dec 08, 2015 11:08 am (10386)

---

Ok, I opted for the constant string that is 100% failsafe.
Basically the modder should:
- set one of the bytes of this hardcoded string in quickbms.exe to 0x00
- upx -9 quickbms.exe
- copy /b quickbms.exe script.bms output.exe

aluigi, posted Thu Dec 10, 2015 10:43 am (10413)

---

QuickBMS 0.7.2 is planned to be released tomorrow.
Please let me know if there is a last-minute fix.

shekofte, posted Fri Dec 25, 2015 10:41 am (10548)

---

Quote:

browser feature !

i have a crazy idea on it
in first approach to your goal , how about to add a extraction report generator for quickbms ?
i mean by option of the user , quickbms instead of extraction of the archive , it just write a text file (may become compressed ) that contains useful information columns (name of file , size , offset ... ) about the content of archive !

for example , various members on web that extracted their own game archives just send (share) their extraction report between each other , so that if someone looking for the 3d model of "BMW car" or the .dds texture of a "sunflower" plant , can easily search through this text based report and explore for the desired archive ... more like an online contribution circle !
by this way we don't have to extract archives of all games on every computer .

aluigi, posted Fri Dec 25, 2015 4:30 pm (10554)

---

do you have some examples of such reports?
for example what's the most common format and what fields are used.

Anyway remember that sometimes quickbms works with memory files and chunked files, so a similar thing is totally useless.
That's also why the browser idea as been abandoned, because it's a process composed by 1 listing and N extractions that it's easy for various game formats but absolutely wrong for the others using custom obfuscations, chunked files and so on.
That's the problem of being a "generic" tool that must work in all the situations.

shekofte, posted Mon Dec 28, 2015 8:56 pm (10590)

---

excuse me sir for late !

Quote:

for example what's the most common format and what fields are used.

when i looked more accurately , i found that the architecture of the report quite be flexible !
we can make the choices deeper and depend on the various file-types only useful fields that are keywords for that file-type will be printed ?
since it is possible by quickbms when an individual file decompressed on RAM just simply inspect the few bytes of header and extract the relevant fields ...

example for a .bmp or "BM" magic key according to this 010 editor binary template , the Width and Height Fields of image are in a fixed offset at 18 and 22 (decimal) :

http://www.sweetscape.com/010editor/templates/

Code:

struct BITMAPFILEHEADER bmfh      0   14   Fg: Bg:0xE0E0E0   
CHAR bfType[2]   BM   0   2   Fg: Bg:0xE0E0E0   
DWORD bfSize   364662   2   4   Fg: Bg:0xE0E0E0   
WORD bfReserved1   0   6   2   Fg: Bg:0xE0E0E0   
WORD bfReserved2   0   8   2   Fg: Bg:0xE0E0E0   
DWORD bfOffBits   1078   10   4   Fg: Bg:0xE0E0E0   
struct BITMAPINFOHEADER bmih      14   40   Fg: Bg:0xE0E0E0   
DWORD biSize   40   14   4   Fg: Bg:0xE0E0E0   
LONG biWidth   607   18   4   Fg: Bg:0xE0E0E0   
LONG biHeight   598   22   4   Fg: Bg:0xE0E0E0   
WORD biPlanes   1   26   2   Fg: Bg:0xE0E0E0   
WORD biBitCount   8   28   2   Fg: Bg:0xE0E0E0   
DWORD biCompression   0   30   4   Fg: Bg:0xE0E0E0   
DWORD biSizeImage   0   34   4   Fg: Bg:0xE0E0E0   
LONG biXPelsPerMeter   0   38   4   Fg: Bg:0xE0E0E0   
LONG biYPelsPerMeter   0   42   4   Fg: Bg:0xE0E0E0   
DWORD biClrUsed   256   46   4   Fg: Bg:0xE0E0E0   
DWORD biClrImportant   0   50   4   Fg: Bg:0xE0E0E0   
struct RGBQUAD aColors[256]      54   1024   Fg: Bg:0xFFFFE0   
struct BITMAPLINE lines[598]      1078   363584   Fg: Bg:   

by this way i guess just by simple functions of quickbms can make such ideal reports that just need the offset of desired fields to predefined for each format (png ,wave . 3ds) or by identify of "MAGIC KEY" for quickbms !

may be the report looks something like this :
in the ascii text each line of it , is a record that belongs to a single file summary ! fields expressed in equation form

Code:

"a0014.arc (ARCHIVE NAME)"
".bmp"   "sunflower(FILENAME)"      Width=607 Height=598 
.
.
.
".wav"   "clap (FILENAME)"       bitrate=44100 Channels=2
.
.
.
".3ds"      "BMW CAR(FILENAME)"       vertices=89 faceindices=136

aluigi, posted Tue Dec 29, 2015 7:43 am (10594)

---

The only information that can be dumped are those you can see with -V
So:
- name of the variable
- offset of the field
- size of the field (like 2 for short)
- value read from that field

Maybe you mean a XML-like version of the -V output?

shekofte, posted Fri Jan 01, 2016 1:34 pm (10667)

---

Luigi Auriemma you must make the year 2016 an Annus Mirabilis

aluigi, posted Fri Jan 01, 2016 7:53 pm (10670)

---

aluigi, posted Fri Feb 05, 2016 8:50 am (11214)

---

I have just found (and confirmed) that the game Warframe includes the Oodle library: https://steamdb.info/depot/230411/
So I will probably add its support in the next quickbms.
If you know other games that include compression libraries and code that are NOT available yet in quickbms, please list them here.

The following are a couple of games that are not covered yet by the comtype algorithms of quickbms:
- WALL-E (PS2), it seems developed by Asobo
- QP: Shooting Dangerous
- Drake & Josh Talent Showdown ???

Other games, some of them already solved, can be found by searching "unknown compression" or "unknown algorithm" site:zenhax.com on google.

aluigi, posted Mon Mar 07, 2016 2:15 pm (11832)

---

Speed.

If there will be ever a quickbms 0.8 (major version), it's priority will be improving its speed because currently the tool is really too slow in some situations (for example chunked files, get/putvarchr, xmath and so on).
This is for sure something I will have to improve.

aluigi, posted Tue Apr 19, 2016 5:36 pm (12706)

---

QuickBMS 0.7.4 is almost ready for the release.

Please let me know if you have something (small) to suggest or fix.

JONATHANTKB, posted Sat May 28, 2016 10:01 pm (13754)

---

aluigi can update unity3d_webplayer cant unpack new files from unity 5.3.4 p1

aluigi, posted Sun May 29, 2016 8:11 am (13758)

---

The reference topic for Unity files is viewtopic.php?f=9&t=12
Anyway, currently supporting new versions of Unity is a problem.

JONATHANTKB, posted Sun May 29, 2016 5:51 pm (13778)

---

ok thanks

Ekey, posted Tue May 31, 2016 5:23 pm (13833)

---

It's posible to add operators (OR / AND) for IDString? In one game with two different publishers in archives only different identifiers. It would be nice to add this. I mean something like:

Code:

IDString "ABCDEFG" or "GFEDCBA"
IDString "ABCDEFG" and "GFEDCBA"

or

Code:

IDString "ABCDEFG" || "GFEDCBA"
IDString "ABCDEFG" && "GFEDCBA"

?