This topic was archived here:

https://web.archive.org/web/20230000000000fw_/https://www.zenhax.com/viewtopic.php?t=7589

hhrhhr, posted Tue Mar 20, 2018 4:29 pm (33476)

---

hpk format:

Code:

// header, 36 bytes
char magick[4]; // "BPUL"
uint unk1;      // 36 (header size or data offset??)
uint unk2[5];   // 1, -1, 0, 0, 1

uint entry_offset;  // absolute
uint entry_size;    // entry_offset   entry_size = EOF

// data
char filedata[xxx]; // FOURCC can be "LZ4 " or other

// file desc
struct dir_entry[xxx] {
    uint ptr;   // entry[ptr] -> next item
    uint attr;  // 0 - directory, 1 - file
    short sz;
    char name[sz]   // name of current dir or file
}

// entry_offset
struct entry[xxx] {   // xxx = entry_size >> 3
    uint offset;        // absolute, pointer to dir_entry (included dir, file or filedata)
    uint size;          // size of dir_entry or zsize of file
}

LZ4 packed file:

Code:

char magik[4];          // "LZ4\x20"
uint size;              // unpacked size
uint block_size;        // 2^17
uint h_size;            // header size; if size==0, then EOF here
uint chunk_offset[xxx]; // xxx = (h_size - 16) >> 2, absolute offsets of chunk begins

struct chunk[xxx] {     // offset = h_size
    char lz_data[];     // full LZ4 block (unpacked == block_size)
}
char lz_data_tail[];    // partial LZ4 block (unpacked < block_size)

for unpack can be used LZ4_decompress_safe (src, dst, zsize, size). if (entry.size == size - h_size) then file content is not packed.

implementation in Lua here.

MerlinSVK, posted Tue Mar 20, 2018 8:12 pm (33479)

---

Or you can use HPK Archiver, modded by Atom0s

https://github.com/atom0s/HpkArchiver

Win build:

HPK_archiver_1_0_14_mod.7z

hhrhhr, posted Tue Mar 20, 2018 9:10 pm (33480)

---

"quick&dirty" BMS-script for unpack lz4 compressed files (I hope that someone optimizes this code ;)

Code:

idstring "LZ4 "
comtype lz4

#### bad example, output is corrupt
get SIZE long
get BLOCK long
get H_SIZE long
get ZSIZE ASIZE
math ZSIZE -= H_SIZE

clog "milky_way_bad.dds" H_SIZE ZSIZE SIZE


goto 4 #### back to begin

#### good example, CRC32 1AA851A4
get SIZE long
get BLOCK long
get H_SIZE long
get ZSIZE ASIZE
xmath CHUNKS "(H_SIZE - 16) / 4"

math I = 0
set CHUNK_OFF[I] H_SIZE
for I = 1 <= CHUNKS
    get CHUNK_OFF[I] long
next I
set CHUNK_OFF[I] ZSIZE

math CHUNKS   1
for I = 1 <= CHUNKS
    math VAR = CHUNK_OFF[I]
    math I - 1
    math VAR - CHUNK_OFF[I]
    set CHUNK_SZ[I] = VAR
next I   2
math CHUNKS - 1

for I = 0 < CHUNKS
    clog "milky_way_good.dds" CHUNK_OFF[I] CHUNK_SZ[I] BLOCK
    if I == 0
        append
    endif
next i
xmath TAIL "SIZE - CHUNKS * BLOCK"
clog "milky_way_good.dds" CHUNK_OFF[I] CHUNK_SZ[I] TAIL
append

zbychos, posted Tue Mar 27, 2018 9:49 pm (33613)

---

Why does the script not save the file under the file's input name?
You can not do it?

rm_mist.tga
0006fada 18 milky_way_good.dds
Info: algorithm 249
offset 0006fada
input size 0x00000012 18
output size 0x00000012 18
result 0xfffffffc -4

Error: the uncompressed data (-4) is bigger than the allocated buffer (131072)

=============================================
Most of the "tga" files work.
In "lua" it only cuts out a few bytes from the beginning of the file.

hhrhhr, posted Wed Mar 28, 2018 8:05 am (33619)

---

zbychos wrote:

Why does the script not save the file under the file's input name?

because this is a test case for a single file. in your case, the input size is equal to the output, that is, no compression is used.

Quote:

In "lua" it only cuts out a few bytes from the beginning of the file.

can an example? the name of the hpk-archive and the "cutted" file?

makc_ar, posted Wed Mar 28, 2018 9:38 am (33620)

---

I found decompressor

Code:

idstring "LZ4 "
comtype lz4
get SIZE long
get CHUNK_SIZE long
get NAME filename
xmath TMP "SIZE % CHUNK_SIZE"
if TMP != 0
   math TMP = 1
else
   math TMP = 0
endif
xmath CHUNKS "(SIZE / CHUNK_SIZE)   TMP"
append
for i = 1 <= CHUNKS
   get OFFSET long
   savepos POS
   if i == CHUNKS
      get ZSIZE asize
   else
      get ZSIZE long
   endif
   math ZSIZE -= OFFSET
   goto POS
   if ZSIZE < CHUNK_SIZE
      clog NAME OFFSET ZSIZE CHUNK_SIZE
   else
      log NAME OFFSET ZSIZE
   endif
next i
append

zbychos, posted Wed Mar 28, 2018 6:07 pm (33626)

---

hhrhhr wrote:

zbychos wrote:

Why does the script not save the file under the file's input name?

because this is a test case for a single file. in your case, the input size is equal to the output, that is, no compression is used.

Quote:

In "lua" it only cuts out a few bytes from the beginning of the file.

can an example? the name of the hpk-archive and the "cutted" file?

hhrhhr
I apologize for my joke but I could not resist ...: D.
I know it's just an example.
OK, the end of jokes.

I used HPK Archiver

graphics catalog - UI
file UI\Icons\Research\rm_mist.tga
It may be a bug in the file but I have hpk once again and it's the same.

Lua 'Data' catalog
It does not work with any * .lua file
see lua before.rar and lua after.rar

Such a request: can you give a way to store the directory structure in the 'output' directory?

get FNAME filename
get FFOLDER input_folder
string NAME = FFOLDER
string NAME = "/"
string NAME = FNAME

It works more or less, but not as I expected. I have never had contact with QuickBMS before and for now I find it difficult to understand the basics.

zbychos, posted Wed Mar 28, 2018 6:12 pm (33627)

---

makc_ar wrote:

I found decompressor

makc_ar
I do not want to worry you, but it's cleansed and improved the same script.
In action nothing changed

makc_ar, posted Wed Mar 28, 2018 6:25 pm (33628)

---

zbychos
Try this viewtopic.php?f=9&t=2630

hhrhhr, posted Wed Mar 28, 2018 10:41 pm (33633)

---

zbychos wrote:

I used HPK Archiver

I'm not the author of this program.

Quote:

Such a request: can you give a way to store the directory structure in the 'output' directory?

if you have in mind the finalization of the bms-script, then I'm not interested. for full unpacking there is the aforementioned HPK Archiver. my lua script on githab (hpk_unpack.lua) also unpacks all lz4 compressed files.

zbychos, posted Fri Mar 30, 2018 1:22 am (33655)

---

OK no problem. Thanx

nuke974, posted Fri Apr 27, 2018 12:34 pm (34411)

---

Someone in the world have figured out how to convert the .hgm file?