This topic was archived here:

https://web.archive.org/web/20230000000000fw_/https://www.zenhax.com/viewtopic.php?t=7204

aluigi, posted Fri Jan 12, 2018 12:38 pm (31480)

---

Just a quick announcement about an alternative reimport mode implemented in QuickBMS 0.8.2a.

Now in the quickbms package you will find a new reimport2.bat file that activates this new mode.
REIMPORT2 is no longer limited by the size and compressed size of the reimported files, if the file is bigger it will be appended at the end of the archive.

The usage is the same of reimport.bat, the only difference for the user is that you will no longer see the usual "infamous" error:

Code:

Error: file "???"
       the reimport option acts as a reimporter and so you cannot reinsert a
       file if it's bigger than the original otherwise it will overwrite the
       rest of the archive or cannot be loaded correctly:

         new size: ? uncompressed
         old size: ? uncompressed

- do you want to skip this file? (y/N/force)

It doesn't mean that your edited archive will work "for sure" with your game, it just offers a way to bypass 2 important limits of the standard reimport mode:

• size of the files
• TOC information with some archives/games, in fact the offset, size and compressed size fields will be rewritten

Technical explanation:

Example of input archive

Code:

###112233################DATADATADATA#######
   | | |                 |
   | | |                 file
   | | size
   | compressed size
   offset

Standard reimport mode

Code:

###112233################DATADATADATA#######
                         |          |
                         | new file |

New reimport2 mode

Code:

###112233################DATADATADATA#######DATADATADATADATADATA
   | | |                                    |                  |
   | | |                                    | new bigger file! |
   | | new size
   | new compressed size
   new offset

As usual you will find all the info and limits (don't expect miracles) in section 3 of http://aluigi.org/papers/quickbms.txt

Feel free to post your positive and negative feedback about reimport2 here in this topic

sigroon365, posted Fri Jan 12, 2018 1:42 pm (31481)

---

Great works! Amazing!

Drawing, posted Fri Jan 12, 2018 4:05 pm (31483)

---

thanks a lot Sensei

Delutto, posted Fri Jan 12, 2018 7:40 pm (31487)

---

Amazing news!!! With this, QuickBMS will become the definitive game moding/hacking tool.
Great work master!

hackspeedok, posted Sat Jan 13, 2018 7:08 am (31502)

---

Amazing !!! Great work. Thanks Aluigi

aluigi, posted Mon Jan 15, 2018 8:36 pm (31582)

---

Not sure if it may be useful, but for the next version of quickbms I have in mind a sort of "install"/"uninstall" option that uses the REIMPORT2 mode.
Let's say you make a translation project that modifies 10 files of the archive and you want to distribute it easily and allow the user to apply it as a "patch" that he can remove if necessary.
Technically that would be easily possible by saving the original offset/size/zsize fields and appending all the new files (even those smaller than the originals) keeping the original files untouched, so it's easy to revert the modifications by restoring the original values and "cutting" the archive to its original size.

Anyway, a part from this idea, currently I'm mainly interested in what games work and don't work with reimport2, just for curiosity and "statistics".
Waiting your WORK / DOESN'T WORK list

*edit*
Right, the archives with a TOC located at the end of the archive (the "tail") can't work in reimport2 with files bigger than the originals since the TOC is no longer located at the end of the archive.
Just for statistics, In my collection of over 1800 scripts about 70 have the TOC at the end of the archive.

Anexenaumoon, posted Tue Jan 16, 2018 5:28 pm (31614)

---

Awesome! So there's no longer a need for that "5GB quickbms" executable?

aluigi, posted Tue Jan 16, 2018 5:37 pm (31615)

---

quickbms_4gb_files.exe is necessary for archives bigger than 4 Gb because it uses 64bit variables instead of 32bit.
It's also necessary in specific scripts that require 64bit operations.

Delutto, posted Thu Jan 18, 2018 3:55 am (31667)

---

aluigi wrote:

Right, the archives with a TOC located at the end of the archive (the "tail") can't work in reimport2 with files bigger than the originals since the TOC is no longer located at the end of the archive.

Well, how about a parameter to set a offset at the end of file to reinsert the content? So, this way we can insert the mod file before the TOC or after the last content of the file.

aluigi wrote:

Anyway, a part from this idea, currently I'm mainly interested in what games work and don't work with reimport2, just for curiosity and "statistics".
Waiting your WORK / DOESN'T WORK list

Unreal Engine 4 PAK files have a TOC in the end of file, so should not work.

aluigi, posted Thu Jan 18, 2018 4:01 pm (31684)

---

Delutto wrote:

Well, how about a parameter to set a offset at the end of file to reinsert the content? So, this way we can insert the mod file before the TOC or after the last content of the file.

The only work-around I can see about the TOC-tails is simply doing reimport2 as-is now and then copying the whole TOC at the end of the file, but it will never work with the archives that write the offset of the TOC at the end (many):

goto -8
idstring "SIGN"
get TOC_OFFSET long
goto TOC_OFFSET

That would work only with archives that don't have a TOC offset, like those in which the starting of the TOC is calculated from the number of files or from the size of the TOC, for example: savepos OFFSET ; xmath OFFSET "OFFSET - (FILES * 8)" ; goto OFFSET.
And even in that lucky rare case there is still the problem that quickbms can't magically guess where the TOC starts and ends.

Long story short: no solution or work-around.

No problems for those archives with TOC at the end and TOC_OFFSET at the beginning since doesn't matter where the TOC and the files are located, that already works.

Ah in the "about 70" scripts I guessed as TOC-tail in my previous post I forgot to calculate those having the TOC offset at the beginning and those using "get OFFSET asize ; math OFFSET - ?" instead of "goto -?", so the number is a bit higher, probably still under 100 which is basically "nothing" compared to 1900 scripts.

shadow_lonely, posted Sat Jan 20, 2018 2:16 am (31709)

---

Nice! That's great!

aluigi, posted Mon Jan 22, 2018 8:33 am (31791)

---

In quickbms 0.8.3 I have started to add a feature that tells the user if the archive is invalid for reimport2, it will be improved over time because currently it simply checks if the scripts uses "goto -NUM" (TOC at the end).

I'm not sure if it's a good idea to zero the bytes where the original file is located in the archive, in theory yes because that space is no longer used and the archive will compress better if you want to distribute it.
I preferred to wait, let me know if you find something wrong in doing that.

I don't think I will implement the patch/unpatch option I mentioned some posts above because currently I have not enough statistics to know if it would be worth or not.

sigroon365, posted Fri Jan 26, 2018 8:36 pm (31949)

---

Test result - Doesn't work on Hitman 2016.

aluigi, posted Fri Jan 26, 2018 9:02 pm (31951)

---

I guess the problem with hitman is caused by:

• ZSIZE which is a result of various "&"
• SIZE coming from a memory file
• HASH may be checked by the game at runtime, but I don't know if it's the hash of name (probably) or file (I doubt)

Sajjad_Rahim, posted Mon Jan 29, 2018 12:49 pm (32017)

---

i just tried REIMPORT2 with the evil within 2 but game still crashing... i think because game use ptr for offset and size and pkr for compressed file. any solution?

sigroon365, posted Thu Feb 01, 2018 4:00 pm (32103)

---

aluigi wrote:

I guess the problem with hitman is caused by:

• ZSIZE which is a result of various "&"
• SIZE coming from a memory file
• HASH may be checked by the game at runtime, but I don't know if it's the hash of name (probably) or file (I doubt)

Thanks for your reply.

Graveyard, posted Sat Feb 03, 2018 6:04 am (32138)

---

Excelent!!!

Delutto, posted Mon Feb 05, 2018 10:51 pm (32228)

---

aluigi wrote:

Waiting your WORK / DOESN'T WORK list

REIMPORT 2 doesn't work with yoyogames.bms, more specifically with STRG(Text) file. (Or am I doing something wrong)
I think Slog is the problem(as descripted in section 3 of quickbms.txt), I tested some binary STRG dump with a following script:

Code:

get STRINGS long
for i = 0    get DUMMY long
next i
for i = 0    get STRING_SIZE long
   math STRING_SIZE   1
   slog "Textos.txt" -1 STRING_SIZE
next i

aluigi, posted Tue Feb 06, 2018 9:04 am (32235)

---

Offset -1 means that there is no absolute offset, so the strings are stored sequentially and therefore it's not possible to store a string longer than the original

GHOST DEAD, posted Sat Feb 24, 2018 11:45 am (32707)

---

in Force mod, archive will work with a new size or not? (for those script can be repack(inject) a file in archive)

aluigi, posted Sat Feb 24, 2018 3:08 pm (32710)

---

doesn't exist "force" in reimport2.
NEVER use "force" in reimport mode.

happydance, posted Wed Feb 28, 2018 7:24 pm (32796)

---

wow, this is really great news!

cant wait to try it out.

aluigi, posted Sat Mar 03, 2018 10:01 am (32854)

---

quickbms 0.8.4 will return a warning if you use reimport2 on sequential content, for example chunked files or files saved with "savepos OFFSET ; log output.dat OFFSET 0x100" and so on. That will avoid to corrupt the archive without giving any explanation to the user since currently (0.8.3) the new data is appended at the end of the file and the game will still read the old content due to the lack of an offset reference (it's sequential).

Additionally it will also zero the original space if the file is bigger, that will save space when you will distribute the compressed edited archive.
I wasn't sure if enabling or not this feature but it offers also the advantage of knowing if something is wrong with the reimport operation, let's say something doesn't work as expected and the game tries to read the old content, it will be just zeroes (invalid) and it's easy to notice it (crash, errors, corrupted images/text).

Hopefully there will be no bugs with these two changes

RC-3197, posted Tue May 08, 2018 5:24 pm (34774)

---

This is an amazing step forward, now if only we could add new files then it would be perfect!