aluigi, posted Fri Mar 30, 2018 11:47 am (33668)

---

I remember that in the past quickbms.exe exported ALL its functions but I don't remember what happened later since lot of stuff was added and compatibility problems arised, anyway using exported functions from an executable may not work with some functions and still requires the developer to have knowledge of the prototype of the function and how to use it... not easy for most .NET developers.

Currently the only "solution" would be to execute quickbms.exe and pass a basic script via argument of the -s option but it's far to be a good solution.

Added to my TODO list

michalss, posted Fri Mar 30, 2018 7:21 pm (33680)

---

aluigi wrote:

I remember that in the past quickbms.exe exported ALL its functions but I don't remember what happened later since lot of stuff was added and compatibility problems arised, anyway using exported functions from an executable may not work with some functions and still requires the developer to have knowledge of the prototype of the function and how to use it... not easy for most .NET developers.

Currently the only "solution" would be to execute quickbms.exe and pass a basic script via argument of the -s option but it's far to be a good solution.

Added to my TODO list

Thx mate, i have not seen code from qbms coz im not kind a c person, but im sure you can figure this out,im mostly using your compressions. Problem with method you mention, is the speed, specially if some files are chunked, it is amost impossible to use it like this..

chrrox, posted Mon Apr 09, 2018 9:54 am (33910)

---

Apple is using new zip method 99.
https://github.com/lzfse/lzfse/blob/mas ... internal.h
sample zip
http://iosapps.itunes.apple.com/apple-a ... j4ykFvA=

aluigi, posted Fri Apr 13, 2018 8:24 am (34012)

---

lzfse is already available in quickbms but apparently something was wrong, in fact even lzfse_compress crashes.
added to my todo list

aluigi, posted Fri Apr 27, 2018 4:16 pm (34417)

---

the problem with lzfse was caused by the version I was using in quickbms, the latest lzfse works correctly without any modification to my code.

quickbms 0.8.5 will have all the functions exported and probably some functions called quickbms_compression, quickbms_compression2 and quickbms_encryption to easily use all the algorithms embedded in the tool without headaches.
but keep in mind that loading an executable as a dll without additional code that reconstructs the import table is a bad idea and sometimes works sometimes doesn't.

Example currently used in my tests:

Code:

    HMODULE hlib = LoadLibrary("quickbms.exe");

    int __cdecl (*quickbms_compression)(char *algo, void *in, int zsize, void *out, int size) = NULL;
    quickbms_compression = (void *)GetProcAddress(hlib, "quickbms_compression");

    output_size = quickbms_compression("zlib", input, input_size, output, output_size);

michalss, posted Fri Apr 27, 2018 6:43 pm (34422)

---

aluigi wrote:

the problem with lzfse was caused by the version I was using in quickbms, the latest lzfse works correctly without any modification to my code.

quickbms 0.8.5 will have all the functions exported and probably some functions called quickbms_compression, quickbms_compression2 and quickbms_encryption to easily use all the algorithms embedded in the tool without headaches.
but keep in mind that loading an executable as a dll without additional code that reconstructs the import table is a bad idea and sometimes works sometimes doesn't.

Example currently used in my tests:

Code:

    HMODULE hlib = LoadLibrary("quickbms.exe");

    int __cdecl (*quickbms_compression)(char *algo, void *in, int zsize, void *out, int size) = NULL;
    quickbms_compression = (void *)GetProcAddress(hlib, "quickbms_compression");

    output_size = quickbms_compression("zlib", input, input_size, output, output_size);

Thx luigi, looks very very nice, ill try some compressions... yeah.. Just is there any C# sample equivalent pls ?

aluigi, posted Fri Apr 27, 2018 6:50 pm (34425)

---

I'm not sure about adding quickbms_encryption mainly because there are many arguments for configuring it and may result chaotic to use.

For example this is the syntax of the quickbms command: Encryption ALGO KEY [IVEC] [MODE] [KEYLEN]

5 arguments plus data and size... 7 in total
Maybe I can opt for a solution like quickbms_compression and quickbms_compression2 where the former is basic to use (no dictionary) and the latter is the full (complex) version.

michalss, posted Fri Apr 27, 2018 7:12 pm (34426)

---

aluigi wrote:

I'm not sure about adding quickbms_encryption mainly because there are many arguments for configuring it and may result chaotic to use.

For example this is the syntax of the quickbms command: Encryption ALGO KEY [IVEC] [MODE] [KEYLEN]

5 arguments plus data and size... 7 in total
Maybe I can opt for a solution like quickbms_compression and quickbms_compression2 where the former is basic to use (no dictionary) and the latter is the full (complex) version.

Encryption will be tricky im sure of it but compression should be straight forward to do.. To be honest arguments are not the problem as i can see, problem would be to feed such a functions with correct data.

aluigi, posted Fri Apr 27, 2018 7:45 pm (34428)

---

Worst case scenario there are over 13250 functions exported from quickbms.exe to use

michalss, posted Fri Apr 27, 2018 7:48 pm (34430)

---

aluigi wrote:

Worst case scenario there are over 13250 functions exported from quickbms.exe to use

He uff that is a lot. BTW can you please share sample of compression use in c# ? Thx

aluigi, posted Sat Apr 28, 2018 8:47 am (34450)

---

Maybe I will provide some examples when 0.8.5 is out.

michalss, posted Sat Apr 28, 2018 10:41 am (34456)

---

aluigi wrote:

Maybe I will provide some examples when 0.8.5 is out.

That would be very nice... Thx let us know pls

aluigi, posted Sat Apr 28, 2018 1:06 pm (34458)

---

Anyway I think the only way is releasing quickbms as a dll otherwise there is no practical way to use its functions.
And currently I don't plan to do that since it requires the release of another file (the big dll) and so on, I simply needed something that required zero effort from my side.

Exist also other possible IPC solutions that may be easier to implement and I'm interested in your feedback.

For example an option can be a TCP socket or (even better) a local webserver that can be used by any program for performing the work like

Code:

POST http://localhost:12345/quickbms_compression?algo=zlib&decompressed_size=12345

input provided as binary content of the POST request and the decompressed data as content of the answer.
the decompressed size is necessary for most of the compression algorithms, "decompressed_size" is just an example of name, "size" or "output_size" would be good too (maybe all of them, since input size is implicit in POST).

That would be 100% independent by programming language, OS and even system since 127.0.0.1 can be replaced by all the interfaces.
Is it easy for you to call web API from .NET?

michalss, posted Sat Apr 28, 2018 9:00 pm (34473)

---

aluigi wrote:

Anyway I think the only way is releasing quickbms as a dll otherwise there is no practical way to use its functions.
And currently I don't plan to do that since it requires the release of another file (the big dll) and so on, I simply needed something that required zero effort from my side.

Exist also other possible IPC solutions that may be easier to implement and I'm interested in your feedback.

For example an option can be a TCP socket or (even better) a local webserver that can be used by any program for performing the work like

Code:

POST http://localhost:12345/quickbms_compression?algo=zlib&decompressed_size=12345

input provided as binary content of the POST request and the decompressed data as content of the answer.
the decompressed size is necessary for most of the compression algorithms, "decompressed_size" is just an example of name, "size" or "output_size" would be good too (maybe all of them, since input size is implicit in POST).

That would be 100% independent by programming language, OS and even system since 127.0.0.1 can be replaced by all the interfaces.
Is it easy for you to call web API from .NET?

well it is not a problem call webapi, but there will be limitation on size during the data transfer i guess.

aluigi, posted Sun Apr 29, 2018 9:59 am (34485)

---

Yeah, right.
Apparently the dll is the only solution.

Using shared memory IPC, which is the only fastest solution, would be a pain for the developers to implement.

Anyway I will implement the web and named pipe IPC interfaces anyway mainly because it's easy and I already have the code, maybe it will be useful in particular situations... after all it requires no effort from my side since the code is already there

aluigi, posted Sun Apr 29, 2018 9:07 pm (34506)

---

Just to confirm that the web IPC is slow indeed and also prone to timeouts due to the time necessary to perform some compressions.
Anyway now it's there and it's something funny that may be useful for the future.
Currently it implements compression, encryption and execution/testing of both an input script and input file.

michalss, posted Sun Apr 29, 2018 10:20 pm (34514)

---

aluigi wrote:

Just to confirm that the web IPC is slow indeed and also prone to timeouts due to the time necessary to perform some compressions.
Anyway now it's there and it's something funny that may be useful for the future.
Currently it implements compression, encryption and execution/testing of both an input script and input file.

Ok can you please post example of usage for c# pls?

sergop, posted Wed May 02, 2018 6:18 pm (34612)

---

Hey, I just got an idea. Can u make option force all when reimporting? It's troubling to do it 300 times.

aluigi, posted Thu May 03, 2018 5:32 pm (34642)

---

@sergop
It's troubling to do it 300 times because you are doing something wrong 300 times, even doing it one single time means corrupting the archive and quickbms warns you about it.
If it was for me I would have not even implemented that "force" choice.

sergop, posted Fri May 04, 2018 2:09 pm (34662)

---

I use it to reimport edited images, videos and text. I have no problems with the games that I used it on just that sometimes there are too many files to import and I have to do it many times.

aluigi, posted Fri May 04, 2018 2:36 pm (34663)

---

As I have already said, you are doing something wrong.

DO NOT USE, NEVER EVER, THE "FORCE" CHOICE

sergop, posted Fri May 04, 2018 3:03 pm (34668)

---

But I have always used it before and I had no problems with any game. I'll try to stop using it, but idk what I'll do without it. Thx for the tip.

aluigi, posted Fri May 04, 2018 4:37 pm (34670)

---

quickbms tells you that it's not possible to reimport the file if its size or its compressed size is bigger than the original file.

In these cases there are two options:
- find a way to make your file smaller
- find a way to reduce the "entropy" of the file, that helps when it gets compressed

As you probably know, few months ago has been introduced reimport2.bat that allows to bypass the size limits in various formats.
That's for sure a good solution if the format is compatible with this mode or it's not possible to use the normal reimport mode (for example because files are compressed with lz4 that fails in decompression).

sergop, posted Sat May 05, 2018 12:10 pm (34700)

---

Yea I think I used -w -r -r a couple of times when -w -r didn't work
Thx for the tip I wasn't exactly sure why it was the case. And most of the time the size is bigger when I reimport the things.

aluigi, posted Mon May 07, 2018 5:19 pm (34752)

---

Let's say that quickbms will have the possibility of loading the script from the clipboard, that would be useful because many people post scripts on forums and websites.
What would be the best way to allow the user to choose the clipboard instead of a real file?
Maybe a command-line option? But in that case nobody is going to use it.
Writing "clipboard" in the entry of the dialog?