aluigi, posted Wed Jul 11, 2018 6:56 am (36677)

---

ok for compression I understand what's the problem, it's exactly what it says so a problem during allocation because the compression "reallocates" the buffers which is something impossible with the buffers provided externally from your tool.
The quickbms_hash variable can't be read from the dll.
Added to my TODO list and will check what I can do in the next release, consider that (re)compression and hashing are not exactly the features I was thinking to for the dll, I made it mainly for the decompression and encryption that you confirmed work correctly.

michalss, posted Wed Jul 11, 2018 12:10 pm (36685)

---

aluigi wrote:

ok for compression I understand what's the problem, it's exactly what it says so a problem during allocation because the compression "reallocates" the buffers which is something impossible with the buffers provided externally from your tool.
The quickbms_hash variable can't be read from the dll.
Added to my TODO list and will check what I can do in the next release, consider that (re)compression and hashing are not exactly the features I was thinking to for the dll, I made it mainly for the decompression and encryption that you confirmed work correctly.

Nice thx for info and btw great job. Main reason to do this is to use compression and decompression and encryption in own tools, so compress is very important in this case as well.

AnonBaiter, posted Mon Jul 23, 2018 8:54 pm (36961)

---

Code:

putarray 0 0 -0.534796
putarray 1 0 -2.33745618
putarray 2 0 -1.77319
putarray 3 1 0.199748
putarray 4 1 1.900874
putarray 5 1 2.44347639
...

let's just assume i'm going to write a bms script that needs this many "arrays"(about 70 ARRAY-s per 20 VAR_IDX-es each, or 1 ARRAY per 80 VAR_IDX-es each if you will) to even do anything - surely i'm not going to write it like this
point is, i need a simplified putarray command so that having to come up with gigantic arrays stored within a bunch of indexes doesn't turn out to be an exercise in patience and frustration

aluigi, posted Tue Jul 24, 2018 3:19 am (36975)

---

Do you mean something like the following?

putarray 0 -1 VALUE
putarray 0 -1 VALUE
putarray 0 -1 VALUE
...
putarray 1 -1 VALUE
putarray 1 -1 VALUE
...
sortarray 0 1 # the last 1 will automatically applying the sorting to all the arrays and not just to array 0

AnonBaiter, posted Tue Jul 24, 2018 4:08 pm (36993)

---

kinda like that yaa

although what i had in mind was more like this

Code:

putarray 0-2 0 -0.534796 -2.33745618 -1.77319
putarray 3-5 1 0.199748 1.900874 2.44347639

aluigi, posted Fri Jul 27, 2018 6:30 am (37039)

---

way too complex

AnonBaiter, posted Fri Jul 27, 2018 10:46 am (37049)

---

its actually pretty simple to me

aluigi, posted Tue Jul 31, 2018 4:21 am (37108)

---

if something is completely different than the expected syntax (PutArray ARRAY VAR_IDX VAR) then it can't be simple at all.

The only space available is after the 3rd argument which is reserved for future usages, anyway quickbms supports a limited number of arguments (a bit less than 32).

shekofte, posted Thu Aug 02, 2018 3:01 pm (37173)

---

I kiss your hands Luigi Auriemma for all of your efforts !
Providing of quickbms as dll must be a new generation !
I ashamed to say you even in my current age i am not properly familiar with programming of codes that insert dll commands !!!
I begging for a very very simple tutorial may be in c that show how to use quickbms DLL ?

aluigi, posted Sat Aug 04, 2018 12:27 am (37195)

---

I posted simple C examples few posts above:
viewtopic.php?p=35965#p35965

shekofte, posted Sat Aug 04, 2018 8:52 am (37206)

---

aluigi wrote:

I posted simple C examples few posts above:
viewtopic.php?p=35965#p35965

acknowledge

michalss, posted Mon Aug 20, 2018 1:01 pm (37698)

---

Hi Aluigi, is there any ETA to fix compression and encryption issues in DLL pls ? Tomb raider is comming and we would like to use it for that.. Thx a lot

aluigi, posted Mon Aug 20, 2018 1:14 pm (37701)

---

Honestly I still don't have a date for the next version of quickbms (and consequently quickbms.dll) but I expect it for the end of September.

Currently that specific problem with the dll is not very simple to handle because the internal functions of quickbms, like the compression/encryption ones, are meant to work with the memory of quickbms while in that dll the memory is provided from outside.

What I mean is that it would be impossible to support some comtypes like unzip_dynamic of lzma_dynamic but I can do 2 things to improve the compatibility:
- disabling the memory protection of quickbms (basically the same that happens when you run quickbms.exe with the -9 option, I didn't check if this is already default)
- avoid reallocating memory when quickbms_compression and quickbms_encryption are used

michalss, posted Mon Aug 20, 2018 9:04 pm (37726)

---

aluigi wrote:

Honestly I still don't have a date for the next version of quickbms (and consequently quickbms.dll) but I expect it for the end of September.

Currently that specific problem with the dll is not very simple to handle because the internal functions of quickbms, like the compression/encryption ones, are meant to work with the memory of quickbms while in that dll the memory is provided from outside.

What I mean is that it would be impossible to support some comtypes like unzip_dynamic of lzma_dynamic but I can do 2 things to improve the compatibility:
- disabling the memory protection of quickbms (basically the same that happens when you run quickbms.exe with the -9 option, I didn't check if this is already default)
- avoid reallocating memory when quickbms_compression and quickbms_encryption are used

Well im sure it is not simple, but im 100% convidence you will manage it as always.. Thx keep us posted..

aluigi, posted Mon Sep 17, 2018 11:44 am (38563)

---

In the next version quickbms will have an option (-U) for listing all the supported compression algorithms.
And when using the comtype scanner it will dump the files with the name of the algorithm instead of their number, yeah no longer annoying navigation if defs.h for locating the algorithm.

aluigi, posted Fri Sep 21, 2018 6:03 pm (38708)

---

Since reimport2 is not limited by the size of the files, I can implement as many fake (aka nop) compressors I desire.

For example some compression algorithms are available only in their decompression implementation lacking the code necessary for recompressing the data, therefore reimporting is not possible.
A fake compressor would bypass the problem.

If interested please let me know what algorithms you would like to see implemented.
Even better provide the code for fake compressor yourself

aluigi, posted Sun Sep 23, 2018 1:11 pm (38751)

---

The fake compressors are a huge success.
I already implemented most of those that were missing in my scripts and they all work:

Code:

DRV3_SRD_COMPRESS
YAZ0_COMPRESS
BIZARRE_COMPRESS
BIZARRE_SKIP_COMPRESS
BLACKDESERT_COMPRESS
DR12AE_COMPRESS
EA_COMP_COMPRESS
LBALZSS_COMPRESS
MOHLZSS_COMPRESS
MOHRLE_COMPRESS
NISLZS_COMPRESS
QCMP1_COMPRESS
RFPK_COMPRESS
RLEW_COMPRESS
SAINT_SEYA_COMPRESS
SHREK_COMPRESS
SLZ_01_COMPRESS
SLZ_02_COMPRESS
SLZ_03_COMPRESS
UCLPACK_COMPRESS
SEGA_LZS2_COMPRESS
WOLF_COMPRESS
YAKUZA_COMPRESS
YKCMP_COMPRESS

Implementing them is really a joke requiring one minute if you have the source code of the decompression algorithm and no bit-operations are involved.

Shokoniraya, posted Sun Sep 30, 2018 12:47 pm (38893)

---

looks like QuickBMS has issue with single files in reimport2
can you just add a tag to replace raw file (not re-overwrite OR write in next data)
i know its a problem in a single file without OFFSET, but a command-option must disable this error and replace raw file, not write. just for single files that no offset are located

if you do that, then we can even make scrip for text and string files!

i really cant reimport2 such files!

aluigi, posted Wed Oct 03, 2018 10:58 pm (39027)

---

In that case you should use reimport.bat and then select "force" when quickbms asks you if you want to continue due to your file being bigger than the original.

There is no issue or bug, reimport.bat and reimport2.bat work exactly as they are meant to do

Shokoniraya, posted Thu Oct 04, 2018 11:15 am (39048)

---

aluigi wrote:

In that case you should use reimport.bat and then select "force" when quickbms asks you if you want to continue due to your file being bigger than the original.

There is no issue or bug, reimport.bat and reimport2.bat work exactly as they are meant to do

i know about force mode, quick bms cant updata file size in force mode and reimport2 cant import bigger file in whit this script
my QuickBMS vesion; 0.9.0 (2018-17 jun)

aluigi, posted Thu Oct 04, 2018 1:30 pm (39054)

---

You don't need quickbms to do something like that and in any case you are doing it in the wrong way because it's not just a single file that you can put back ignoring all the data that was available after it and the other fields referring to sizes and offsets.
You have at least 2 size fields at offset 0x90 and 0x94 and the data from offset 0x351b is NOT the ogg file, it's part of the archive format/container that you are going to screw by overwriting them.
It's extremely important to never overwrite data that doesn't belong to the extracted file and why reimport and reimport2 work in that way.

Shokoniraya, posted Fri Oct 05, 2018 7:09 am (39085)

---

aluigi wrote:

You don't need quickbms to do something like that and in any case you are doing it in the wrong way because it's not just a single file that you can put back ignoring all the data that was available after it and the other fields referring to sizes and offsets.
You have at least 2 size fields at offset 0x90 and 0x94 and the data from offset 0x351b is NOT the ogg file, it's part of the archive format/container that you are going to screw by overwriting them.
It's extremely important to never overwrite data that doesn't belong to the extracted file and why reimport and reimport2 work in that way.

Aluigi, then READ THIS

in this case, plase read this note carefully

i want to say some things, real and important. im gonna tell you some Property that Must add into QuickBMS.exe

1_Importing In Bigger Size;
as you can see, quick bms is verry good for anyone, who wants starting to editing files and archive. Admittedly, need To Improt Bigger Size in Files
im not tlking about reimport2 for set new offset and new size! im talking about memory_size or those files without offset!

2_Multi Size:
in my Previous post, my file had multi size,
i cant see any tag in QuickBMS.exe for update both size.
some archve has Archive_Size, like UMD (Unreal Engine 2.5) and for import2, Archive_size must turn to read base size!
also some archives has decompressed_size and compressed_size, and we cant do it with Quick bms, and thats not well for someone like me that cant make a new tool or dont know any other programing language Like C# or C and even C.
why QuickBMS? (for any other user who wants do the same without learning a real programming language. [Aluigi Auriemma]) remember? that was some good words from you

3-Single_Raw
i read some post from some poeple, like (SIMS 4 STBL). if you adding a tag to disable write and do replace Raw_File(in this case, user cant import one file, that means user must extract all and import all, not just import some file[for update all size]. and you must do it in reimport2, not force mode. or maybe, should add a command-option to enable force mode in reimport2 mode, that would be awsome and its a big Process for quickbms! [just for text files and single files])

4-log file
i wish i could had a log file to read and keep my history of quickbms useage, its useul for some guys like me.

aluigi, posted Fri Oct 05, 2018 8:15 am (39093)

---

1) unfortunately quickbms is not a magician and can't store 100 bytes in a place for 99 bytes. The only way to do that is the presence of an offset field that can be replaced with a new one pointing to the new data.

2) compressed and uncompressed sizes are fully supported in reimport2 when the Clog command is invoked, usually you have a compression flag in the archives that tells you if a file is compressed and I bet this is probably the case. It's up to you to understand the format and figure out what are the flag, compressed and uncompresse size fields.

Just for curiosity (not related to this case so ignore this phrase once avoiding confusion) there is only one rare case in which reimport2 can't work properly and it's when there is no compression flag, for example just a "if SIZE == ZSIZE ..." check but even in that case it's possible to make the script compatible by using comtype copy and clog instead of log.

3) honestly I have not understood but it looks something related to question 1 or you forgot to remove the non-edited files from the folder, quickbms reimports only the available files. While in extraction it's possible to filter the extracted files based on their names and extensions so if you have 1000000 files and only 10 jpg images you want to edit, it's better to use the -f "{}.jpg" option avoiding to extract 999990 useless files.

4) it depends by what type of log you need.
Launching quickbms with the -V option returns tons of useful technical information about the input data.
The -v option is rarely useful and it's related to the commands of the script.
They must be used from command-line maybe with the "less" cygwin tool (suggested) or with a simple redirect to file:

Code:

quickbms.exe -V -0 script .bms input_file.dat > log.txt
quickbms.exe -V -0 script .bms input_file.dat | less

Shokoniraya, posted Fri Oct 05, 2018 10:00 am (39103)

---

i totally get it, aluigi! thank you for explain!
and i just want to tell you: can you add force mode in reimport2?
because in reimport1 force, file size cant update...
half of problem solved in force mode, but if you add force mode in reimport2 too, i can solve that!

aluigi, posted Fri Oct 05, 2018 4:22 pm (39122)

---

reimport2 can't have a force mode because if the file is bigger it's automatically added at the end (without any questions to the user) and its offset/size fields rewritten.

I guess that what you want is a way for automatically replacing the offset/size fields and writing the file without caring what's the format, sort of "mix" of reimport reimport2.
In that case it's better if you write a script for specifically doing that (not easy since it's not the job of quickbms) or just do it manually with a hex editor.

reimport.bat does not replace the original size field for compatibility with any format and for compatibility with the original behaviour of such feature: overwriting just the bytes of the file and nothing else.