aluigi, posted Thu Sep 03, 2020 11:55 am (58518)

---

I still miss the purpose behind the name of the temporary files created by quickbms.
They are immediately deleted after the launched executable terminates and the content of the output file is read by quickbms.

I found and fixed the bug of BMS_FOLDER.

LZX is already available since years and in many variants.
Comtype:
- lzx, the version used in an old tool
- xmemdecompress, the most used in games
- lzxcab, used in the Microsoft CAB files (there is also a _delta version)
- lzxchm, used in the Microsoft CHM files (there is also a _delta version)
- cabextract, same as lzxcab I guess
- DS_LZX

The compressor is already available for xmemdecompress and ds_lzx.

Regarding the last question it's a bit complicated.
The library used by quickbms doesn't return the original name and even in that case I would have to choose between the original URL and the name of the file provided by the server.
Since quickbms is NOT a downloader and it only needs the content of the file, it's more appropriate to keep the original URL.
What I will do in the next version is improving the available names, example:

Code:

filename     file.php
fullname     https://zenhax.com/files/file.php_id=9002
basename     file
extension    php
fullbasename download/file
filepath     download

quickbms has the same download speed of anything else.
Probably you are comparing it with downloaders downloading the file with multiple connections which is totally unrelated.

Shokoniraya, posted Sun Sep 06, 2020 6:08 am (58558)

---

sir aluigi, one other thing

can you update QuickBMS guess function for file extension? like mp3 files, or adding more guess for media files

aluigi, posted Sun Sep 06, 2020 9:03 pm (58563)

---

MP3 files don't have an header since they are just a sequence of compressed frames without any "magic" value (often they start with byte 0xff but it's useless).
Some years after mp3 has been made it started to be used the ID3 struct for containing metadata like artist, name of the song and so on.
QuickBMS uses the ID3 "magic" for guessing the mp3 extension, so there is no extension if there is no ID3.

As far as I know quickbms already supports most of the format available, if something is not supported it's just because there is not "magic" to guess.

Let me know if you have suggestions about a specific format

Shokoniraya, posted Sat Oct 17, 2020 3:03 am (59520)

---

found a problem
QuickBMS asks -w in slog exporting (tested with last beta)

here a script
viewtopic.php?f=12&t=14240&p=59519#p59519

i've attached a sample file too (in this comment)

aluigi, posted Thu Oct 22, 2020 4:52 pm (59676)

---

Fixed

Shokoniraya, posted Sun Oct 25, 2020 3:59 am (59734)

---

problem solved (now, im sure UltraEdit broked my file, gotta leave it)

khachai, posted Mon Nov 02, 2020 10:31 am (59905)

---

I got an error like this, please help fix it
https://i.imgur.com/QEh6WDa.jpg

Shokoniraya, posted Tue Nov 03, 2020 1:38 am (59926)

---

sir aluigi, in tested QuickBMS in these days and i can say it's working great

but there is one problem, deflate
zlib/deflate decompression works fine, but looks like some games and even some programs can't read deflate or zlib files that compressed with QuickBMS
such as limbo game, the evil within 1, 7z Gui and some other games that using zlib or deflate
also, if you can use same version of deflate that used in winrar in QuickBMS, it could be great, i compressed files with winrar (in zip mode) and used it and worked fine in those games
(sorry for my bad grammar)

can you replace current version of deflate library of QuickBMS with a safe version one or a public one? i think you should use this: https://zlib.net/zlib1211.zip

aluigi, posted Tue Nov 03, 2020 10:26 am (59934)

---

Wait wait wait.
What you mean with winrar?

winrar creates zip and rar archives.
quickbms extracts data and can't create archives, it can only reinject compressed data back in the original archive.
Obviously a zip after reimport mode will never work due to the different crc.

the_evil_within.bms is a different case because it's not a real deflate, it's customized and indeed the only way to dump that data was using deflate_noerror that ignores errors.

P.S.: quickbms uses only zlib in reimport2.bat, the zlib->zopfli->kzip sequence is only used in reimport.bat for fitting in the original space.

Shokoniraya, posted Tue Nov 03, 2020 6:24 pm (59940)

---

i compressed raw file with winrar, and then, i extract compressed part of zip file that created with winrar (in PK) with hex editor by hand, and used it in those games and works fine, but compressed raw file that QuickBMS creates not working on some games

and in other test
i compressed a .txt file file with winrar to zip archive (with deflate 0x08 flag), now i have a normal compressed zip file
and then, i compressed same .txt file with QuickBMS to deflate and replaced it with compressed part of zip file with hex editor, and i updated zip details with hand too, such as crc32, size or compressed size and offsets (pk offset and table offset at the end of file) and it fails in extraction!

and i repeat same job, but in this time, i used 7z to compress deflate except QuickBMS (creating a zip file in 7z Gui) and after doing same job, it worked with no fail!

i did same thing on the evil within with 7z compressed deflate raw file (again, compressed raw file, not archive! because i cut compressed part of zip file by hand and pasted it to my raw file) and worked!
also, i repeated same job for the evil within with winrar compressed raw file and works file
but when i used deflate of QuickBMS, game crashed!
so if it;s a customized deflate, so why winrar and 7z compressed files worked fine?
QuickBMS deflate and zlib should be replaced with a correct one

i think it was a miss understanding in your reply, because i'm talking about compressed file, not archive, i just used winrar to get compressed raw file

aluigi, posted Wed Nov 04, 2020 8:38 am (59946)

---

Ah cool

So you can do the same test with quickbms too using the following 3 scripts:

Code:

comtype kzip_deflate_compress
get SIZE asize
clog "kzip.dat" 0 SIZE SIZE

Code:

comtype ZOPFLI_DEFLATE_COMPRESS
get SIZE asize
clog "zopfli.dat" 0 SIZE SIZE

Code:

comtype deflate_compress
get SIZE asize
clog "zlib.dat" 0 SIZE SIZE

Please let me know what of the created dumps does work and NOT work.

Ah, just a note, I found that quickbms uses advance_comp for zlib and deflate, not zlb because it gives better ratio.

Shokoniraya, posted Wed Nov 04, 2020 2:51 pm (59953)

---

deflate fails in The Evil Within, but i found out that new winrar versions of winrar can decode it as well, problem is old file i guess. some games using old library (i'm using a old version of winrar, becuase it's faster)
kzip_deflate_compress fails too, compressed file is same as deflate

but ZOPFLI_DEFLATE_COMPRESS worked in both game and winrar!

and one other thing, zlib still fails in games, there is a dll that called Ionic.Zip and it's working on most of games, can you add it to QuickBMS, sir aluigi?

aluigi, posted Thu Nov 05, 2020 5:29 pm (59978)

---

Is that problem happening only with The Evil Within?

How did you test the data compressed by quickbms with Winrar?

Shokoniraya, posted Thu Nov 05, 2020 5:38 pm (59979)

---

winrar compressed binary works on the game, but QuickBMS not
and there is same problem with zlib too

can you add a official version of those compression libraries?

aluigi, posted Thu Nov 05, 2020 5:53 pm (59981)

---

If the problems happen only with that game... there are no probems
That game is a problem itself since even the zlib library returns errors with its compressed data.

So do you have problems with other games? If yes, what games?

I don't know what you mean with "official versions" since zlib and deflate are just standard algorithms with various implementations of different libraries and some settings.
AdvanceCOMP uses code from the old 7-zip and is used in MAME and other software:
https://en.wikipedia.org/wiki/AdvanceCOMP
zlib.dat is produced by its old version since now it relies on zopfli.

Shokoniraya, posted Fri Nov 06, 2020 2:36 am (59989)

---

some games using customized zlib, i think thats the reason

and can you add a reserved variable for reimport mode?
if reimport enabled, then IMPORT_MODE = 1

Code:

if IMPORT_MODE = 1
math OFFSET x 2048
endif

and can you add a command similar to log, but just for injecting binary into the file?

Code:

inject NANE OFFSET SIZE

(with injecting from MEMORY_FILE support too)

im not talking about reimport features
just a simple command for injecting binary, no reversing or imporing and variable operators, to use in export mode too, please accept it
it could be verry useful
i know that it's possible to do it in reimport mode, but it's good to have similar thing in export mode too
your program is a powerful generic tool. this is really needed
thank you sir aluigi

aluigi, posted Fri Nov 06, 2020 8:34 am (59997)

---

What games?

Now I use a different library for compressing data instead of advancecomp, I uploaded only quickbms.exe here:
https://aluigi.altervista.org/beta/quickbms_exe.zip
Please use the zlib.dat script and let me know if that's ok.

Currently I don't see how I can implement something like that import_mode/inject thing for various reasons.

Shokoniraya, posted Fri Nov 06, 2020 10:00 am (59998)

---

game like LIMBO, The Evil Within 1 & 2, ps2 game such as ghost rider, ps3 like life is strange, splinter cell, valiant heart
except The Evil Within and LIMBO, rest of them can works fine with ZOPFLI

and about that command, it's really useful to have a injector on export mode, for creating a file from scratch or some file editing if we want to do it with hand totally, sometimes i need to dump some files when i write a importer script, and there is same story when i want to use a exporter, but i just want to export a file and replace a binary at same time, it just really annoying, i just have to choose between import or export at one script, i get this problem many times and i have to make same copy of script with different name and just for a little change, and when i want to update a part of code, i have to edit many other bms files too, maybe you seeing this as a unnecessary command, but it helps a lot to have some option in exporting and importing at once, i know reversing is not easily in such case, but i'm talking about a command without any reverse features and just for injecting binary within a file with increasing and enlarging input file
many times i had to make same script just for a little change
please accept it sir aluigi, i know you can done this with making same functions of reimport3

dump NAME OFFSET SIZE
inject NAME OFFSET SIZE

aluigi, posted Fri Nov 06, 2020 1:38 pm (60004)

---

So is the zlib.dat generated by the new quickbms.exe ok with those games?
It's a different library.

For the "inject" command I doubt I can do anything because what you pasted means:
dump NAME OFFSET SIZE: I guess you mean log, so extract the file on the output folder
inject NAME OFFSET SIZE: get the file NAME from the output folder (???) and reimport it
Doesn't have any sense in this way, it would dump and reinject the same file and in any case the files are dumped and taken from the same output folder.

Then why should you change the scripts?
The reimport mode is made just to not change any script.

Shokoniraya, posted Fri Nov 06, 2020 2:26 pm (60005)

---

you got this totally wrong sir aluigi, that not what i meant


dump {NAME} {OFFSET} {SIZE}
a command that able to dump files even if reimport mode enabled

inject {NAME} {OFFSET} {SIZE}
to injecting file without any reimport features, just like opening hex editor and replace some byte
(no size or offset block rewriting/updating)

REIMPORT_MODE
a reserved variable, set to 1 if reimport mode enabled, if not, then set to 0


http://www.mediafire.com/file/ci1lbcjli ... p.zip/file
please download this file, it will clear everything, i know it will be a unused command in many cases, but it's not! please and please add these 3 options to QuickBMS
i know reimport mode created for reversing and maybe it's look like that i'm saying this like a child and i didn't think on many things, but that's not true
we can have reversing and exporting options at one script (reversing and editing, not just importing)
QuickBMS is so automatically, that's what i meant, i hope you understand with take a good looking at pk_injection_crc.bms

aluigi, posted Fri Nov 06, 2020 2:45 pm (60006)

---

Wow, what a huge mess

How do you plan to use an "inject" command?
It would take the file from the output folder, not much sense.

I checked the scripts you provided and definitely you need to use a programming language.
Scripts are meant to be something simple with just few lines.

Shokoniraya, posted Fri Nov 06, 2020 3:08 pm (60007)

---

i can do it with, i know what i'm saying! names can be done with _temp folder! if there is no output, so that means i can use current dictionary of bat file as well

http://www.mediafire.com/file/ahd74kr34 ... t.zip/file

The Evil Within still crashes and limbo works fine

aluigi wrote:

Scripts are meant to be something simple with just few lines.

i can do it in that way
i can do inject thing too, but take many times and so much memory works and my laptop is weak!
please accept it sir aluigi, that could be useful in many cases

aluigi, posted Sat Nov 07, 2020 4:15 pm (60033)

---

Just for personal testing I added a Reimport command.
It works like the Append command:

Code:

reimport 0: disable
reimport 1: like reimport.bat
reimport 2: like reimport2.bat
reimport 3: the reimport 3 mode
reimport: enable/disable the current mode

Obviously the files are taken from the output folder so they must already be there.

No idea if it can be useful at all but who knows.
Please let me know what you think.

For the normal writing of data in the input file I ever suggest to use getdstring putdstring.

Beta:
https://aluigi.altervista.org/beta/quickbms_exe.zip

Shokoniraya, posted Sat Nov 07, 2020 9:12 pm (60039)

---

like append? what a perfect idea!
i did some little tests and thats what needed to make a hand work script (hand work, i mean without many automatically things and with more options to have, sorry for my limited grammar)
putdstring works fine too

i think it can be useful for saving time and make a script with many options, like a real program!
and it can be better if there was a command line option to make reserved byte with value 1
for example: quickbms.exe -command "NOCRC,JUST_PRINT" -o -w file.bms file.dat

Code:

if NOCRC != 1
put CRC_VALUE long
else
get LAST_CRC_BLOCK long
endif

if JUST_PRINT != 1
log NAME OFFSET SIZE
else
print "%NAME%"
endif

i attached my test

my_test.zip

aluigi, posted Sun Nov 08, 2020 8:33 am (60049)

---

That's already possible with -s:

Code:

quickbms.exe -s "math NOCRC = 1 ; math JUST_PRINT = 1" script.bms output_folder