This topic was archived here:

https://web.archive.org/web/20230000000000fw_/https://www.zenhax.com/viewtopic.php?t=1663

Panzerdroid, posted Tue Oct 27, 2015 9:21 pm (9003)

---

Hello,

It seems that with new version dev changed deflate in game packs to LZO.

Could anyone help with a proper BMS script?

Thanks in advance!

3 small sample files:
http://www43.zippyshare.com/v/TfVlMyeE/file.html

aluigi, posted Tue Oct 27, 2015 9:59 pm (9004)

---

I don't know what compression algorithm is used for the information table at the end of the archives.
And no, lzo is not used for the files (for example icons.pak from offset 0 to 83268)

Ekey, posted Tue Oct 27, 2015 10:16 pm (9006)

---

Well header now always at the end in every pak.

Code:

struct PAKHeader
{
   uint32_t   dwVersion; // 13 > Previous version is 10
   uint32_t   dwTableOffset;
   uint32_t   dwTableSize;
   uint32_t   dwFlag; // 1
   uint8_t    iMD5[16]; // Seems md5 hash
   uint32_t   dwUnknown; // Always 40
   uint32_t   dwID; //LSPK
};

Code:

goto dwTableOffset;
get dwTotalFiles long;

aluigi, posted Tue Oct 27, 2015 10:50 pm (9007)

---

Is there any difference between the Enhanched Edition and the standard one?
I have noticed that the archives of the previous versions were different with the header at the beginning and without this strange compression.

Panzerdroid, posted Tue Oct 27, 2015 11:00 pm (9008)

---

aluigi wrote:

Is there any difference between the Enhanched Edition and the standard one?
I have noticed that the archives of the previous versions were different with the header at the beginning and without this strange compression.

Yes, they redone a lot of things, changelist is huge. So I think changing of pak format was necessary for them (now the game ported to consoles as well).

Ekey, posted Tue Oct 27, 2015 11:05 pm (9009)

---

In previous version table was not compressed, but file data's was compressed by LZ4 and ZLIB. Seems table compressed even as LZ4.

aluigi, posted Tue Oct 27, 2015 11:11 pm (9010)

---

I have read that there is an official editor from Larian for the previous version so maybe it will happen the same for this new one too.

Panzerdroid, posted Tue Oct 27, 2015 11:14 pm (9011)

---

Yeah, maybe some day - I didn't found anything about near plans

Ekey, posted Tue Oct 27, 2015 11:18 pm (9012)

---

It's LZ4, no doubt. PseudoCode (http://i.imgur.com/CBjCg2C.png)

EE_PseudoCode.rar

Panzerdroid, posted Tue Oct 27, 2015 11:30 pm (9014)

---

I slightly missed with compression type
How do you people even understand all those machine codes!

aluigi, posted Tue Oct 27, 2015 11:31 pm (9015)

---

lz4 for the files but lz4 doesn't work with the information table.

Ekey, posted Tue Oct 27, 2015 11:43 pm (9016)

---

Tomorrow i will test it fully.

Ekey, posted Wed Oct 28, 2015 10:01 am (9020)

---

I did a test decompressor (LZ4 r128) and it works like charm

test1.dat - table from Engine.pak
test2.dat - table from Main.pak

*.dec files already decompressed data.

Decompressed size = entry count * 280

DivinityLZ4Test.rar

Ekey, posted Wed Oct 28, 2015 11:05 am (9022)

---

Code:

struct PAKEntry
{
   uint8_t    iFileName[256];
   uint32_t   dwOffset;
   uint32_t   dwZSize;
   uint32_t   dwSize;
   uint32_t   dwPakNum;
   uint32_t   dwUnknown; // Always 66
   uint32_t   dwCRC32; // For compressed data
};

aluigi, posted Wed Oct 28, 2015 12:03 pm (9023)

---

It's a fault of lz4 that returns errors if there are bytes after the stream, just like happens with Witcher3 in reimport mode.
In short I had the 0x0d000000 long still attached to my input data and it caused the error.
I have to check if I can bypass this issue in quickbms to improve the compression scanning or if there is no solution.

aluigi, posted Wed Oct 28, 2015 12:16 pm (9025)

---

Ok, I don't think there is solution for lz4... it's just a raw stream.

The script is almost ready but EngineShaders.pak gives some problems.
*edit* there is an additional 32bit zsize at the beginning, the files are too small to guess if it's a tentative of chunks or just a different and useless way.

Additionally does someone know if some of the paks of the previous versions were compressed too?

Panzerdroid, posted Wed Oct 28, 2015 12:51 pm (9026)

---

Test decoder works, thank you! But I guess it doesn't support decompression of the whole pack yet, right?

aluigi, posted Wed Oct 28, 2015 12:55 pm (9027)

---

This is the script but we need to find how it should work with EngineShaders.pak because lz4 fails with the second file:

*edit*
http://aluigi.org/bms/lspk.bms

Ekey, posted Wed Oct 28, 2015 1:09 pm (9028)

---

Here example pak's from previous version (all types)

Ekey, posted Wed Oct 28, 2015 1:24 pm (9031)

---

Also seems some pak files could be separated. For Example Textures.pak have normal header at the end but Textures_1.pak and Textures_2.pak it does not have. here samples.

aluigi, posted Wed Oct 28, 2015 1:44 pm (9034)

---

Thanks a lot, I have updated the script.
It should work also with multi paks.
Remains just the problem with EngineShaders.pak.

happyend, posted Thu Oct 29, 2015 12:12 am (9054)

---

Script 0.2 not support Sound.pak,Sound_1.pak.and SoundBanks.pak

RageX, posted Thu Oct 29, 2015 5:41 am (9059)

---

happyend wrote:

Script 0.2 not support Sound.pak,Sound_1.pak.and SoundBanks.pak

It also doesn't seems to extract Textures, Textures_1, Textures_2.pak files.

Panzerdroid, posted Fri Oct 30, 2015 4:22 pm (9101)

---

Bump