Space Marine 2 - Swarm Engine - SDC: Shader Definition Cache

[GameBreaker]

The Swarm Engine SDC file is basically a cache of shader groups, their entries and each group's bulk compressed data, that when uncompressed, contains the bulk data of the group which the entry then has reference for the size of each entry's uncompressed data in the then uncompressed bulk data, which is each entry's DXBC shader ( bytecode, not source ).

The hierarchy:
- Header
     > Group Count
     > Entry Count
     > Bulk Data Size
     > Sanity Check
- Group
     > Group Bulk Data Size ( Compressed )
     > Group Entry Count
     - Entry
          - Entry Hash ( CRC32 )
          - Entry Bulk Data Size ( Uncompressed )
          - EntryStringLength
- Group Bulk Data ( Based On An Group's Compressed Size )
     - Entry Bulk Data ( Based On An Entry's Uncompressed Size ) ( DXBC Shader / Data Files / Unknown 4 Byte Files )

ImHex Pattern
```

import std.core;

struct SDCHeader {
    u32 GroupTotal;
    u32 EntryTotal;
    u32 BulkDataSize;
    u32 SanityCheck;
    };
    
struct SDCEntry {
    u32 EntryHashCRC32;
    u32 EntryUncompressedSize;
    u32 EntryStringLength;
    };
    
struct SDCGroup {
    u32 GroupBulkDataSize;
    u32 GroupEntryTotal;
    SDCEntry Entry [GroupEntryTotal];
    };

struct SDCGroupData {
    u8 ZLibCompressedData [parent.Group[std::core::array_index()].GroupBulkDataSize];
    };

struct SDCFormat {
    SDCHeader Header;
    SDCGroup Group [Header.GroupTotal];
    SDCGroupData BulkData [Header.GroupTotal];
    };
    
SDCFormat Format @ 0x00;
```
Python Script Included For Extraction Of Data Files, might not be as up to date with the findings above.
Thank you again, Jumanji144.

Command in Powershell:
python3 scp.py <sdc file> <output folder>

Thank You:
- Jumanji144 ( ImHex Discord )
- WerWolv ( ImHex Discord )
- Haus ( Halo Archive Discord )

sdc_working.py

Edited September 21 by GameBreaker
Addition of python script command.

[shak-otay]

Thanks.

But (other than for the hierarchy structs) it would make more sense to attach the python code as a py file rather than displaying the code in the post, too, imho.

Also having test.scd as a sample would be nice. 

Edited September 18 by shak-otay

[GameBreaker]

8 minutes ago, shak-otay said:

Thanks.

But (other than for the hierarchy structs) it would make more sense to attach the python code as a py file rather than displaying the code in the post, too, imho.

Also having test.scd as a sample would be nice. 

Apologies, doing this between my laptop and remote-ed in desktop, so easier to copy and paste text in this case, but will edit this post as I get this all put together in files.

[GameBreaker]

On 9/18/2024 at 3:30 PM, shak-otay said:

Thanks.

But (other than for the hierarchy structs) it would make more sense to attach the python code as a py file rather than displaying the code in the post, too, imho.

Also having test.scd as a sample would be nice. 

Included the python script now, and some changes are made to the pattern file, thanks to Haus, the script is not fully up to date with the pattern, but it should be mostly fine for research purposes.