aluigi, posted Sun Jun 28, 2015 10:36 am (5967)

---

I have checked that file and I think it's impossible that you get that error.
The reason is that witcher3.bms performs a sequential extraction of the files in the cache archives (even if there is a sort of complex table at the end which is ignored) so you can't get a 0% coverage.
You may get that error only when quickbms has extracted all the files and reaches the final header, but in that case you should have already an over 95% of coverage and not 0%, and you should have over 4gb of extracted files on disk.

For example here I can extract all the files available in the part you provided extracted almost 3 megabytes of files.

How many files have you extracted from texture.cache?

Fig8man, posted Sun Jun 28, 2015 4:19 pm (5979)

---

9,146 files totaling to .98 gb
and its all .dat files and files with extensions I've never heard of (.a8a,.b0b,.bxm,.bxp,.c1a,.cia,Etc)

aluigi, posted Mon Jun 29, 2015 1:31 pm (5992)

---

Ok that's a problem because it's not the whole file.
An alternative is using offzip that will be able to decompress the whole archive but the names of the files will be the same.

The fact that the script doesn't parse the final header and information is probably caused by differences in the structure of different cache files, I don't remember why I don't parse it but usually that's the reason.

Fig8man, posted Mon Jun 29, 2015 4:12 pm (6001)

---

someone from the nexus forums was kind enough to upload the extracted texture.cache files to mega

Code:

Here: https://mega.co.nz/#fm/nVVxlJ4S

(But beware its BIG! over 13,000 files! the only way i was able to download the whole thing was by importing it into my MEGA Cloud and downloading it with MEGA's sync client for windows)
i found the gwent cards i was looking for but for some reason they're really blue in color so i guess the gwint_game.redswf module is doing some kind of color manipulation on its end after it calls the .dds

hhrhhr, posted Mon Jun 29, 2015 6:40 pm (6006)

---

Fig8man wrote:

But beware its BIG! over 13,000 files! the only way i was able to download the whole thing was by importing it into my MEGA Cloud and downloading it with MEGA's sync client for windows

did you try script for manual textures unpack?

Fig8man wrote:

i found the gwent cards i was looking for but for some reason they're really blue

can you write their filenames?

Amioran, posted Tue Jul 07, 2015 10:19 am (6192)

---

Luigi, I tried to contact you by email but for now you have not replied me.

I have a problem with using QuickBMS and the witcher3.bms to reimport some specific files (and only them) as when you try to reimport them the file size is too large even when the character count and file size is the same as the extracted source (you can either reimport the source file intact and it will do the same). It is like when exported the file becomes larger than the source or the reimporting compressing algorithm doesn't work as it should.

One example is in patch.bundle inside gameplay\abilities\monster_abl_base.xml; if you export the file somewhere and then if you try to reimport it inside the bundle even with no change at all the reimport will say that the file is too large.

Only certain extracted files have this problem (usually large files) while for all the rest it works flawlessly. I used QuickBMS to reimport changed files in the Witcher 3 bundles without problems now for almost a month, but now I'm stucked because there are certain files (as I said usually large ones) that have this problem and cannot be reimported correctly.

Do you think there could be a solution to this issue?

aluigi, posted Tue Jul 07, 2015 1:02 pm (6197)

---

Hi, yes I remember your email and I also replied to it.
It was about lz4 and reimporting.
The problem is unclear because the other users didn't report issues with the latest quickbms and script.
Anyway the following is my original that hopefully is still valid for solving your problem:

Code:

Hi,

can you check if you are using quickbms 0.6.4?

It may be possible that these specific files are compressed with lz4.
The lz4 code used in versions before 0.6.4 wasn't able to compress the
files good like the code used in the game (lz4 has been improved a lot
recently) and that caused a difference of few bytes that didn't allow to
reimport some files back.

Anyway there is a worst problem with lz4, the library returns an error
if the size of the new compressed data is minor than the original one
because it handles also the bytes between its end and the original
limit.
The full details are available here:
http://zenhax.com/viewtopic.php?p=5500#p5500

Just in case it may be helpful, recently I updated also the script to
version 0.1.2 for adding another compression algorithm used by the game:
http://aluigi.org/papers/bms/others/witcher3.bms

You may also like to post on the zenhax forum in case other people have
the same problems or have found a solution.

Amioran, posted Wed Jul 08, 2015 7:39 am (6214)

---

Updating to 0.6.4 and the new witcher3.bms resolved the issue, thanks!

I didn't think about checking up if there was an update of quickbms to resolve the problem, my bad.

Anyway, thank you very much for taking the time to reply, now I can finally alter some specific things I couldn't before for my mod.

EDIT:

I did rejoice too soon. The reimporting doesn't give an error now, but it corrupts the bundle anyway. Using the aforementioned "monster_base_abl.xml" and reimporting it as it is (i.e. just extracting the file and reimporting it back) in the patch.bundle will corrupt it: all monsters will have level -1 with no health bar and unkillable. At first I thought I made an error somewhere while modifying the file, but then I made a test with the original source (and to be sure starting from an original backup of patch.bundle) and the error persisted. Not reimporting just that specific file fixed the issue (everything worked fine).

So, albeit now the reimporting process doesn't give errors there's still something wrong in the compression somewhere (or some other problem, you are surely much more knowledgeable than me on the matter to understand the possible cause) because when using those files (even without any change whatsoever from the extracted source) will corrupt the bundle when reimporting them back.

To be sure I tried 5 different times, with different files and starting always from an intact bundle source. The first two tries were with "monster_base_abl.xml" in patch.bundle (under gameplay\abilities). Extracted it, reimported, loaded save file with a group of wolves in the near vicinity, all of them had -1 level and unkillable. The third try was with "def_item_weapons.xml" in the same patch.bundle (under gameplay\items). Did the same thing, loaded save game and most weapons were vanished altogether from the inventory. The fourth and fifth try was with "def_item_weapons_secondary.xml" in the xml.bundle (under gameplay\items). Same procedure, loaded save game and all secondary weapons were gone from the inventory (as before because much probably unrecognized).

As for the problem with the file size that cannot be less than the original one, I did already know. Even a single character difference and reimporting the file will corrupt the archive. You must be sure that the file size is exactly the same (and it's an hassle many times).

aluigi, posted Wed Jul 08, 2015 10:37 am (6218)

---

That's the problem explained in the second paragraph. It's how the lz4 library works and currently there is no solution, it's independent from quickbms.

Amioran, posted Wed Jul 08, 2015 11:29 am (6220)

---

aluigi wrote:

That's the problem explained in the second paragraph. It's how the lz4 library works and currently there is no solution, it's independent from quickbms.

Ok, I understand.

I guess that at this point the only solution is a rebuilder. There's already a way to create bundles using the Witcher executable but the way the metadata.store works is not yet fully understood and there's no way to modify it. Right now is a complete mess to recreate the metadata.store and then redistributing it would require from users to do that same mess.

EDIT:

I have a strange idea, actually. Right now Quickbms checks the uncompressed file size, but if you could also check the compressed one then you could, for example, add characters to the source in forms of comments to reach the same exact compressed length, hence resolving the issue of the garbage being added (in the previous versions you could actually see the compressed size of the file you were about to reimport). In little words: if you could add an option that instead of checking the uncompressed file size it checks the compressed one and, knowing how many bytes are needed to fill the gap, then you could easily add comments in the file to resolve the issue (a thing that you must do in many cases anyway when you have a file size that's lower than the original because you removed some line, for example). It would be the same exact thing only done for the compressed size instead of the uncompressed one and in an "exact" way (i.e. enabling the reimport only if the *compressed* file size is exactly the same).

If I'm saying an idiocy please excuse me, as I'm not very versed in compression algorithms, however it is similar to hex editing yourself the LZ4 compressed file to remove the garbage and simply add 00s to reach the same size. But I can understand that doing all this mess just for a single game is a bit nonsensical (this also in the case my idea would actually work to begin with).

aluigi, posted Wed Jul 08, 2015 12:20 pm (6221)

---

You can't add data to the new file before recompressing it because the game may not appreciate it.
It's like reading a text file with 4kb of zeroes at the end, and it's worst if it's a specific format that reads every single byte of the file including the new added ones.
Additionally it means that a sort of brute forcer is necessary.
quickbms is a general tool so it must follow rules that work in any situation.

That's also the reason why it's not possible to modify the original compressed size value, simply because nobody knows where it comes from. It can be a 32bit field read from the file or from a memory file or calculated with a math operation or just a fixed number... anything.

What quickbms already does in reimporting is adding zeroes after the new file to match the size of the original file (line 1801 of src\file.c).

Doggystyle, posted Wed Jul 15, 2015 10:01 pm (6377)

---

., can we translate the subtitles? Is it possible?

I want to translate this game to my own language but i can't find any tool about it..

aluigi, posted Thu Jul 16, 2015 7:31 am (6388)

---

Personally I don't know where are located the subtitles, maybe someone else knows that.

Haoose, posted Fri Aug 14, 2015 11:44 am (6859)

---

The Witcher 3: Wild Hunt now with the mod support
http://thewitcher.com/news/view/1094

Download Witcher_3-Modkit:
http://filedelivery.nexusmods.com/Misc/ ... da68d1a8af

SLaYeR983, posted Mon Aug 31, 2015 2:22 am (7045)

---

aluigi wrote:

Personally I don't know where are located the subtitles, maybe someone else knows that.

in -Game dir\content\contentx... just w3strings files...

Decode and encode systems similar like Gibbed Red Tool (for Witcher2). We waiting for your support...

ner0, posted Sun Jan 17, 2016 9:57 pm (10838)

---

aluigi wrote:

You can't add data to the new file before recompressing it because the game may not appreciate it.
It's like reading a text file with 4kb of zeroes at the end, and it's worst if it's a specific format that reads every single byte of the file including the new added ones.
Additionally it means that a sort of brute forcer is necessary.
quickbms is a general tool so it must follow rules that work in any situation.

That's also the reason why it's not possible to modify the original compressed size value, simply because nobody knows where it comes from. It can be a 32bit field read from the file or from a memory file or calculated with a math operation or just a fixed number... anything.

What quickbms already does in reimporting is adding zeroes after the new file to match the size of the original file (line 1801 of src\file.c).

This is a really annoying problem...
For example, I'm trying to tweak an available mod to my personal taste by changing values related to an NPC inside def_item_weapons.xml
After reinjecting the files what happens is that usually the game doesn't like the values I've put in and the result is that the NPC loses it's weapon.
So I keep trying and trying until it works without actually understanding what makes it work or not work.
Here's an example of 2 lines from the original def_item_weapons.xml:

Code:

If I change the values in the second line from 220 to 250, the compressed and uncompressed byte count is the same but the game will let me know that something is not right by stripping the NPC of it's weapon. Now, if I change the same values to 240, then it will work. I can't figure out an exact logic but the repetition of numbers will influence the outcome.
If I do the opposite it also works, I change the values in the 1st line from 250 to 240 and the values in the 2nd line from 220 to 250. Another example, changing the values in the 2nd line from 220 to 210 will also fail, but keeping this change and changing some 2 other lines somewhere else in that xml from "0.10" to "0.40" will make it work again. This is pretty weird...
So the compressed/uncompressed size is not all, what else could there be? I mean, quickbms shows both sizes and they're always consistent both when it works and when it doesn't.

Savage, posted Wed Jun 15, 2016 1:06 am (14413)

---

Hi! im trying to extract the xbm files from the textures.cache,the script works in some way i only get files like 00000.dat 0000002.dat, at the end of the .cache we have the filenames

Quote:

haracters\models\monsters\endriaga\model\h_04__endriaga_a01.xbm characters\models\monsters\endriaga\model\h_04__endriaga_d01.xbm characters\models\monsters\endriaga\model\h_04__endriaga_n01.xbm characters\models\monsters\endriaga\model\h_04__endriaga_s01.xbm characters\models\monsters\endriaga\model\t_02__endriaga_spiral_a01.xbm characters\models\monsters\endriaga\model\t_02__endriaga_spiral_d01.xbm characters\models\monsters\endriaga\model\t_02__endriaga_spiral_n01.xbm characters\models\monsters\endriaga\model\t_02__endriaga_spiral_s01.xbm characters\models\secondary_npc\auberon\model\body_01_ma__auberon_a01.xbm characters\models\secondary_npc\auberon\model\body_01_ma__auberon_d01.xbm characters\models\secondary_npc\auberon\model\body_01_ma__auberon_n01.xbm characters\models\secondary_npc\auberon\model\body_01_ma__auberon_s01.xbm environment\architecture\elven\common\elven_bridge\textures\elven_bridge_a.xbm environment\architecture\elven\common\elven_bridge\textures\elven_bridge_a_n.xbm environment\architecture\elven\common\elven_bridge\textures\elven_b

I already tried this lua script (wich works -after some compilatons in Linux-, but no idea how to import the files)
https://github.com/hhrhhr/Lua-utils-for-Witcher-3 , maybe it gives some idea.

I uploaded a 81.18mb's file (textures.cache) to see if it's possible to extract the full xbm files with the folders/path
http://www27.zippyshare.com/v/qin8AnGx/file.html

And a extracted texture using the lua script (268kb's) from the example uploaded above using the lua script
http://www27.zippyshare.com/v/8XtM7Z3e/file.html

I remember we had two scripts for the Witcher 2

w2_dds2xbm.bms

Quote:

open FDDE BIN 0
open FDDE DDS 1
get h_size long #size of header
append
log memory_file 4 h_size #write header
append
goto h_size
get tmp long #skip
savepos offset
get mips long #mipmaps
get tmp1 long #width
get tmp2 long #height
get tmp3 long #maxsize
get mip_size long #size of mipmap 1
set dds_offset long 0x80
#print "%offset% %dds_offset% 0 - %tmp1% %tmp2% %tmp3% %mip_size%"
append
log memory_file offset 20 #write header of mipmap 1
log memory_file dds_offset mip_size 1
append
for i = 1 math dds_offset = mip_size
savepos offset
get tmp1 long #width
get tmp2 long #height
get tmp3 long #maxsize
get mip_size long #size of mipmap %i%
#print "%offset% %dds_offset% %i% - %tmp1% %tmp2% %tmp3% %mip_size%"
append
log memory_file offset 16 #write header of mipmap %i%
log memory_file dds_offset mip_size 1
append
next i
append
set memory_file2 binary "\x00\x00\x00\x00\x00"
log memory_file 0 5 memory_file2 #write 00000
append
get size asize memory_file
get name BASENAME
string name = .xbm
log name 0 size memory_file

and
w2_xbm2dds.bms

Quote:

set MEMORY_FILE binary "\x44\x44\x53\x20\x7C\x00\x00\x00\x07\x10\x0A\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x20\x00\x00\x00\x05\x00\x00\x00\x44\x58\x54\x31\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x08\x10\x40\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
set MEMORY_FILE2 binary "\x00\x00\x00\x00"
findloc c_type string "TCM_"
goto c_type
getct c_type string 0x8C
#TCM_DXTNoAlpha,TCM_Normals - DXT1
#TCM_DXTAlpha, TCM_NormalsHigh - DXT5
get name basename
if c_type == "TCM_DXTAlpha"
putvarchr MEMORY_FILE 0x57 0x35 byte #DXT5
string name _DXT5.dds
elif c_type == "TCM_NormalsHigh"
putvarchr MEMORY_FILE 0x57 0x35 byte #DXT5
string name _DXT5.dds
else
string name _DXT1.dds
endif
findloc start string \x3a\x5c
goto start
get null string
getdstring null 0x06
get mips long
get width long
get height long
get MaxSize long
get size long
savepos offset
set h_size offset
math h_size -= 20
putVarChr MEMORY_FILE2 0 h_size long #size of header
append
log MEMORY_FILE2 0 offset
append
putVarChr MEMORY_FILE 0xC height long
putVarChr MEMORY_FILE 0x10 width long
append
log MEMORY_FILE offset size
append
#print "%i% - %offset%, %width%x%height%, %maxsize%, %size%"
for i = 1 math offset = size
append
log MEMORY_FILE2 offset 16
append
goto offset
get width long
get height long
get MaxSize long
get size long
#print "%i% - %offset%, %width%x%height%, %maxsize%, %size%"
savepos offset
next i
get size asize MEMORY_FILE
log name 0 size MEMORY_FILE
string name - .dds
string name .bin
get m_size asize MEMORY_FILE2
log name 0 m_size MEMORY_FILE2

Maybe it works with textures of The Witcher 3? i have no idea..

Anyway, Thanks!

aluigi, posted Mon Nov 07, 2016 5:59 am (18624)

---

Script 0.1.3 for supporting the cache files version 2.

traderain, posted Mon Feb 13, 2017 8:25 pm (20813)

---

Can someone please try and update this for the Witcher 3? Sadly the xbm2dds and dds2xbm does not work. I would be really gratefull if someone did this.