aluigi, posted Sun Oct 21, 2018 6:42 pm (39774)

---

I guess that feature has been never tested since there were at least 2 bugs in it
Will be fixed in 0.9.1

StreamThread, posted Tue Oct 23, 2018 3:34 pm (39803)

---

Does this mean that also will fixed a allocation error, appearing when 'log' many files into Memory file with final size more then 1 gb and next try to 'log' this?

aluigi, posted Wed Oct 24, 2018 2:12 pm (39823)

---

It depends by the error because it looks like you are trying to decompress (clog) a big file and therefore quickbms must allocate a double amount of memory.
Let's say you have already extracted various files and the current one has a compressed size of 900Mb and an uncompressed size of 1.1Gb, that means:
- current memory (let's say 500Mb) 900Mb 1.1Gb

You can try using the -9 option of quickbms but it shouldn't change much.
Anyway it's not a bug or similar.

For the log command (without encryption) quickbms doesn't allocate memory.

StreamThread, posted Wed Oct 24, 2018 6:57 pm (39829)

---

It once was in my one packing script with log command, no clog. Was used 3 memory files: for files table, for files data, and for process loaded files in some order. At the last stage, table and data paste together and export. No one file was compressed, but yes, to memory file with files table encryption was assigned.

aluigi, posted Fri Oct 26, 2018 4:09 pm (39873)

---

So the problem was related to 3 big memory files that weren't able to be allocated in memory.
Unfortunately the memory is very limited, let's say less than 3Gb (it's a 32bit application).

bugmenot, posted Tue Oct 30, 2018 11:02 pm (39981)

---

aluigi wrote:

bugmenot wrote:

I think I found a bug with the -P command or the Rpa Renpy Nopython script:

When extracting an RPA file that contains Cyrillic characters in some of the file names they did not extract correctly so tried adding -P UTF8 (unless I'm wrong RPA files are always UTF-8 encoded) but the result was still wrong, next I tried every other UTF and Cyrillic option from this table using both Identifier and .NET Name values, I also tried adding the codepage "utf8" and codepage 1251 string in the BMS script with the same result.

For example the correct file name is: Armature-??? ??_0 as can be seen in the attached file table, but the files are extracted as: Armature-?'????' ???????_0, Armature-??N? ????_0 or Armature-?N??? ?N???_0.

This bug is the same reported in viewtopic.php?f=11&t=4797 and I confirm that it will be fixed in the upcoming 0.8.1 version that will be released this week

It seems the problem is still present in v9.0, any suggestions?

For example the HEX string 70 68 6F 6E 65 20 2D 20 D0 BA D0 BE D0 BF D0 B8 D1 8F 2E 70 6E 67 is saved as phone - ????????.png instead of phone - ?????.png
Tested with GDPC and RGSSAD scripts on files containing Russian and Japanese file names.

aluigi, posted Wed Oct 31, 2018 7:51 am (39983)

---

It's an utf8 name which is correctly handled by quickbms, in fact my output name is perfect.

Script used for the test, and tested also with reading of name from file:

Code:

set NAME binary "\x70\x68\x6F\x6E\x65\x20\x2D\x20\xD0\xBA\xD0\xBE\xD0\xBF\xD0\xB8\xD1\x8F\x2E\x70\x6E\x67"
log NAME 0 0

You can get that wrong filename you specified only if you use -P 1251 which indeed is wrong because the correct charset here is utf8.

bugmenot, posted Wed Oct 31, 2018 7:21 pm (40001)

---

In other words you are saying quickbms is running as non-Unicode program that's assuming it's running under 1250 encoding instead of checking the local system non-Unicode settings and taking them in account?

P.S. After testing the provided sample I get a file named "phone - ?????.png" which probably means there is some other problem when reading the file name from the archive.

I've attached a small test GDPC sample file.

aluigi, posted Wed Oct 31, 2018 7:35 pm (40003)

---

Eh? No, quickbms uses utf8 by default, which is the codepage used for the string you provided.

Both the gdpc and rgssad have no hardcoded codepage in them and they don't even use any "unicode" conversion like "get NAME unicode" or "set NAME unicode NAME".
The scripts with hardcoded codepage (932) are: brave_soul.bms, ddworks.bms, hunex.bms and norn9.bms.

If you are experiencing that problem, please tell me how to replicate it step-by-step and upload the files you are using.

There is no problem reading filenames from archive, you can test it with a simple script like the following on a file containing the raw filename like the one you provided:

Code:

get SIZE asize
getdstring NAME SIZE
log NAME 0 SIZE

bugmenot, posted Wed Oct 31, 2018 7:59 pm (40005)

---

Steps to reproduce:
Drag the GDPC.bms over quickbms.exe v9.0
select the asteroids.pck file from previous post
click save

Testing on a Win7 VM with non-Unicode language set to 1251. Switching to English (USA) produces a file named "????????N?.png".

P.S. The second sample is working fine as well.

P.P.S. It seems the problem is related to filenames that contain a directory in them. If I use a file containing "?????.png" the second example is working fine, but if it contains "res://?????.png" I get the wrong name.
"1/?????.png" produces the correct name but "1?????/?????.png" results in "1????????\????????.png"

aluigi, posted Wed Oct 31, 2018 8:47 pm (40006)

---

I confirm it's a bug because if you remove "res://" the name is correctly dumped.
I'm investigating.
Thanks for the feedback.

aluigi, posted Wed Oct 31, 2018 9:41 pm (40010)

---

The problem is not directly related to quickbms but in Windows not liking multiple consecutive path delimiters.

For example:
- WORKS: "res/file.png"
- DOES NOT WORK: "res//file.png" or "res\\\\\\file.png"

Basically it's a problem happening probably only the 0.1% of times but I'm happy you found it

It will be fixed in 0.9.1

Ah I have seen your last edit, the problem with the utf8 folder is known and will be fixed too.

Shadow, posted Wed Oct 31, 2018 11:22 pm (40012)

---

aluigi wrote:

Report any quickbms bug and error here.

The problems must affect quickbms and not specific scripts, for which you can use the Game Archive section.

Before reporting ANY problem be sure to have the latest version of quickbms and the latest version of the script that is causing the problem.

*edit* added tags in the title to avoid OT

bugmenot, posted Thu Nov 01, 2018 5:33 am (40015)

---

aluigi wrote:

The problem is not directly related to quickbms but in Windows not liking multiple consecutive path delimiters.

For example:
- WORKS: "res/file.png"
- DOES NOT WORK: "res//file.png" or "res\\\\\\file.png"

Basically it's a problem happening probably only the 0.1% of times but I'm happy you found it

It will be fixed in 0.9.1

Ah I have seen your last edit, the problem with the utf8 folder is known and will be fixed too.

Thanks

usernameqwerty, posted Sat Nov 17, 2018 5:42 pm (40451)

---

Hi, I'm trying to extract embb's from Shadow of Mordor with shadow_of_mordor.bms 0.4 and I'm getting this error.

Code:

quickbms.exe shadow_of_mordor.bms Patch_1.arch05 som

QuickBMS generic files extractor and reimporter 0.9.0
by Luigi Auriemma
e-mail: [email protected]
web:    aluigi.org
        (Jun 17 2018 - 11:49:55)

                   quickbms.aluigi.org  Homepage
                            zenhax.com  ZenHAX Forum
                     @zenhax @quickbms  Twitter & Scripts

- open input file C:\Users\limited\Downloads\quickbms\Patch_1.arch05
- open script shadow_of_mordor.bms
- set output folder som

  offset   filesize   filename
--------------------------------------
  00000000 0          bundles/player_load_skin01.bndlxml05
  0000304c 65536      bundles/player_load_skin01.bndlxml05

-------------------
*EXCEPTION HANDLER*
-------------------
An error or crash occurred:

*EH* ExceptionCode      c0000005 access violation
*EH* ExceptionFlags     00000000
*EH* ExceptionAddress   016E73E9
                        01210000   004d73e9 quickbms.exe
*EH* NumberParameters   00000002
*EH*                    00000000
*EH*                    0F62EA45

Last script line before the error or that produced the error:
  183 clog NAME OFFSET CHUNK_ZSIZE CHUNK_SIZE

Code:

quickbms.exe -f "*.embb" shadow_of_mordor.bms Patch_1.arch05 som

QuickBMS generic files extractor and reimporter 0.9.0
by Luigi Auriemma
e-mail: [email protected]
web:    aluigi.org
        (Jun 17 2018 - 11:49:55)

                   quickbms.aluigi.org  Homepage
                            zenhax.com  ZenHAX Forum
                     @zenhax @quickbms  Twitter & Scripts

- filter string: "*.embb"
  *.embb
- filter   1: *.embb
- open input file C:\Users\limited\Downloads\quickbms\Patch_1.arch05
- open script shadow_of_mordor.bms
- set output folder som

  offset   filesize   filename
--------------------------------------
  00000000 0          bundles/player_load_skin01.embb
  00003c80 65536      bundles/player_load_skin01.embb

-------------------
*EXCEPTION HANDLER*
-------------------
An error or crash occurred:

*EH* ExceptionCode      c0000005 access violation
*EH* ExceptionFlags     00000000
*EH* ExceptionAddress   016E73E9
                        01210000   004d73e9 quickbms.exe
*EH* NumberParameters   00000002
*EH*                    00000000
*EH*                    0E87330F

Last script line before the error or that produced the error:
  183 clog NAME OFFSET CHUNK_ZSIZE CHUNK_SIZE

I extracted loose files from the same arch05 file with arch05explorer, but it can't extract whole embbs.
Do you still have an old shadow_of_mordor.bms script, like 0.2?

chrrox, posted Sun Nov 18, 2018 1:20 am (40459)

---

zstandard with the -e option worked great on most files in star ocean it fails the last chunk of this file.

never mind they used a 00 size for uncompressed data.

aluigi, posted Mon Nov 19, 2018 1:29 pm (40501)

---

@usernameqwerty
It looks like the identified compression is not xmemdecompress and so it crashes.
That's OT here ("not games") but feel free to post a sample file here:
viewtopic.php?f=9&t=5080&start=60

@chrrox
ok I have updated the slz script to version 0.2.5.
quickbms 0.9.1 will no longer need -e

aluigi, posted Thu Nov 22, 2018 8:51 am (40598)

---

0.9.1 has been released

HPAndroid, posted Fri Nov 23, 2018 2:10 pm (40660)

---

I encountered this error when extracting game's archive:

Code:

QuickBMS generic files extractor and reimporter 0.9.1
by Luigi Auriemma
e-mail: [email protected]
web:    aluigi.org
        (Nov 21 2018 - 14:40:39)

                   quickbms.aluigi.org  Homepage
                            zenhax.com  ZenHAX Forum
                     @zenhax @quickbms  Twitter & Scripts

- GUI mode activated, remember that the tool works also from command-line
  where are available various options like folder scanning, filters and so on

- select BMS script. type ? for using the content of clipboard like a script
- select input archives/files, type * for the whole folder and subfolders
- select output folder where extracting files
- open input file C:\Users\Q.Viet\Downloads\Compressed\mod\main.136.com.ea.game.fifa14_row.obb
- open script C:\Users\Q.Viet\Downloads\Compressed\mod\fightnight.bms
- set output folder C:\Users\Q.Viet\Downloads\Compressed\mod\136

  offset   filesize   filename
--------------------------------------
  19c5fe60 182724     data/gui/locale/matchday/JPN_JP.loc
*** stack smashing detected ***:  terminated

-------------------
*EXCEPTION HANDLER*
-------------------
An error or crash occurred:

*EH* ExceptionCode      c000001d illegal instruction
*EH* ExceptionFlags     00000000
*EH* ExceptionAddress   013927A5
                        00C90000   007027a5 quickbms.exe
*EH* NumberParameters   00000000

Last script line before the error or that produced the error:
  92  log FNAME OFFSET SIZE

Press ENTER or close the window to quit

Sorry! But you can delete this my topic: viewtopic.php?f=9&t=8938
I was confused, you had this topic. Thank you!

aluigi, posted Sat Nov 24, 2018 10:23 am (40681)

---

There are few users who are reporting this exact problem with "stack smashing detected" and I'm investigating.
Here on all my testing platforms (linux, mac, win98, win7, win10, win2003 and so on) I have not been able to replicate it even with the same samples used by who reported the problem, so I hope only a small number of people are affected.
I will give you some updates soon.

aluigi, posted Sat Nov 24, 2018 10:46 am (40684)

---

I will also check why 0.9.1 now takes 100 Mb of RAM more than 0.9.0 (about 280 Mb vs 180 Mb).
It's probably related to some new libraries (maybe the Capstone disassembler).
Investigating...

aluigi, posted Sat Nov 24, 2018 11:12 am (40687)

---

@HPAndroid
Can you provide some information about your system like available RAM and version of operating system?
And can you check if you have that problem even when you run quickbms from command-line?

Example:

Code:

quickbms.exe C:\Users\Q.Viet\Downloads\Compressed\mod\fightnight.bms C:\Users\Q.Viet\Downloads\Compressed\mod\main.136.com.ea.game.fifa14_row.obb C:\Users\Q.Viet\Downloads\Compressed\mod\136

aluigi, posted Sat Nov 24, 2018 11:16 am (40688)

---

Ok I can replicate it
It's caused by something in the folders, the idea came when I have seen the paths you used in your test.

Current work-around:
Use a short folder name like c:\quickbms instead of your home folder.

aluigi, posted Sat Nov 24, 2018 11:26 am (40689)

---

Ok problem found.
It was a stupid bug in make_dir where an useless small buffer was used to contain the absolute path of the folder.

aluigi, posted Sat Nov 24, 2018 4:38 pm (40700)

---

I have fixed everything and found also the reason of the 280 Mb... lz4x.c and lz4x_new.c, yes that code is hunger of memory because it's meant to be a stand-alone tool and not a library but still lot of memory for absolutely nothing.
Now the amount of total RAM taken by quickbms is... 60 Mb

Ah quickbms 0.9.2 will be released tomorrow.