Darkspore File Parser

[shak-otay]

Haha, you really thought that greedy beast would be satisfied then? - No, it demands more and more:

api-ms-win-core-datetime-l1-1-1.DLL

api-ms-win-core-localization-obsolete-l1-2-0.DLL

api-ms-win-appmodel-runtime-l1-1-2.DLL

[JeanxPereira]

40 minutes ago, shak-otay said:

Haha, you really thought that greedy beast would be satisfied then? - No, it demands more and more:

api-ms-win-core-datetime-l1-1-1.DLL

api-ms-win-core-localization-obsolete-l1-2-0.DLL

api-ms-win-appmodel-runtime-l1-1-2.DLL

damn hahaha

yea I think we found what we were missing:
api-ms-win-appmodel-runtime-l1-1-2.dll

I've been trying to look it up on the internet, but it's a bit tricky
 

12 hours ago, shak-otay said:

And dalkon is gone for good without a trace?

yes, since 2022 any attempt to contact via discord, email, social media has failed


edit:
I find it in the wine libraries for linux, I don't know if this would work

api-ms-win-appmodel-runtime-l1-1-2.zip

Edited February 7 by JeanxPereira

[shak-otay]

Well, they will work in wine on linux but that's some "VM crossover", so I really wouldn't expect them to work on native windows.

(And we don't want to run Testing under Linux, do we?)

No, the answer might be here Virtual DLL names, I think.

edit: gosh, I fed it with some renamed dlls (maybe a mix of 32 and 64 bit) and got

error bad exe format
status invalid image format

But the first error is like so (entrypoint not found)"

Testing exe has 64 bit, but your fibers and localization dlls are 32 bit. Maybe that's an issue.

 

Edited February 7 by shak-otay

[JeanxPereira]

45 minutes ago, shak-otay said:

Well, they will work in wine on linux but that's some "VM crossover", so I really wouldn't expect them to work on native windows.

(And we don't want to run Testing under Linux, do we?)

No, the answer is Virtual DLL names, I think.

oh, i think this explains why it was using/searching for "temporary dlls / virtual dlls" from windhalk on x64dbg before I uninstalled it.
before i used Windows 11 with Windwalk, a Windows mod that allows customizations in the system without modifying the files directly, it uses global injection and hooking method etc... none of this matters now

Our intention was to use the parser integrated into the recap project, it would be included with other tools such as extracting .package files and also decompiling .lua files. But due of this damn executable we will leave everything ready with the xml itself (if we can use it xD)

what can we do in this case of virtual DLLs?

Edited February 7 by JeanxPereira

[shak-otay]

I would have to read about virtual dlls but I've lost too much time now (being very busy with my own projects).

See my latest post update - maybe deliver all dlls in their 64 bit version?

If the "entrypoint not found" error vanishes then it was a 32/64 bit issue.

I found appmodel-runtime here. (32 bit version!)

(Renamed it to ...1-2 which is weird, and may cause an error, too.)

edit: I've installed VC_redist.x64.exe (24 MB) and there were no missing dlls any more. But still an "entrypoint not found" error.

Testing still doesn't log anything but you could step through the code for is_player (see pc_el_rogue.noun) to see how it is handled.

I got stuck with memmove and didn't find code which logs to a file (fwrite or so).

Maybe it logs to a buffer in the RAM?

Edited February 10 by shak-otay

[JeanxPereira]

On 2/7/2025 at 3:08 PM, shak-otay said:

is_player (see pc_el_rogue.noun)

great, thanks a lot again! is_player represents a bool on .noun file

Since I didn't make any progress with Testing.exe I tried to improve the new parser I was creating, but I think replicating what this parser does as it does seems to be quite complicated, Dalkon did a good job on it (except for this part where it doesn't work xD)

I think making Testing.exe work seems to be the best option to do, below is what I discovered about the format itself:
 

I managed to create a script in ImHex pattern language to correctly read/map Noun files, which allowed me to improve the new parser written python. However, some data still doesn't make sense—specifically, the values found in a .noun file provided by Dalkon:

<affectsNavMesh>false</affectsNavMesh>
<dynamicWall>false</dynamicWall>
<hasLocomotion>false</hasLocomotion>
<locomotionType>0</locomotionType>
<hasNetworkComponent>true</hasNetworkComponent>
<hasCombatantComponent>true</hasCombatantComponent>
<hasCameraComponent>false</hasCameraComponent>
<spawnTeamId>0</spawnTeamId>
<isIslandMarker>false</isIslandMarker>

Using the Darkspore executable (and the Dalkon Testing.exe parser) in IDA, I can locate references to these data. However, I still haven't found a consistent pattern that works for all the .noun files I've tested. Additionally, I haven't managed to implement hash reading in the files. On recap_server, it appears that the program calculates strings or values for some hashes in the XML generated by the parser. Judging by the order in which the parser reads the data, these hashes appear to indicate the offset of each key (string). My parser currently sequentially searches for null-terminated strings at a fixed address (0x1E0) and assigns them to data types such as modelKey, playerClassData, creatureThumbnailData, etc.

Furthermore, I still need to interpret other file types with structures different from the Noun files (more so than I initially expected, xD), making the overall process quite complex. Dalkon's work on this parser was very good—it's a pity that his effort was lost due to an unknown error in the executable

Noun.hexpat for ImHex.zip

[JeanxPereira]

On 2/7/2025 at 3:08 PM, shak-otay said:

edit: I've installed VC_redist.x64.exe (24 MB) and there were no missing dlls any more. But still an "entrypoint not found" error.

I will try reinstalling all the VC packages again to see if I can get the same result
 

During this analysis I did in Testing.exe I managed to decompile that large function I was talking about earlier: sub_140006470

It took a little time but it was decompiled, I don't know if it can be useful in our analysis

sub_140006470.zip

[shak-otay]

7 hours ago, JeanxPereira said:

During this analysis I did in Testing.exe I managed to decompile that large function I was talking about earlier: sub_140006470

It took a little time but it was decompiled, I don't know if it can be useful in our analysis

From my experience top down approaches with such a big function don't help too much.

You can dive into sub functions such as

v1119 = sub_140001A50(v5129);

and hopefully they don't call another dozens of functions, but usually they do.

For me a bottom up approach makes more sense, such as a breakpoint on the memmov function, then use "continue 'til return".

Where memmove is not the best example, admitted. I'll try fwrite (or similar) asap.

"Your" big function is very good to have an overview and helps structuring thoughts, which is important, too.

[JeanxPereira]

On 2/7/2025 at 3:08 PM, shak-otay said:

VC_redist.x64.exe (24 MB)

could you send it to me? I tried reinstalling the packages but it keeps indicating missing DLLs

[shak-otay]

vc_redist64

But I should mention that I have api-ms-win-core-fibers-l1-1-1.dll

and api-ms-win-core-localization-l1-2-1.dll in the system32 folder (they're from the zip you sent, iirc.)

Also there is api-ms-win-core-localization-obsolete-l1-2-0.dll

(where I simply inserted the string -obsolete into the original dll's name).

[JeanxPereira]

On 2/11/2025 at 5:53 AM, shak-otay said:

I'll try fwrite (or similar) asap.

hey! any news?

[shak-otay]

Well, I have some other project I'm very busy with. Did the packages work for you?

I've still that "entrypoint not found" error as I wrote. So some basic functionality might be missing and it

would be important if someone (you 😄) could make it work without that error.

[JeanxPereira]

On 2/21/2025 at 11:19 AM, shak-otay said:

Bem, tenho outro projeto com o qual estou muito ocupado. Os pacotes funcionaram para você?

Ainda tenho aquele erro "entrypoint not found" como escrevi. Então, alguma funcionalidade básica pode estar faltando e

seria importante se alguém (você😄) poderia fazê-lo funcionar sem esse erro.

hey! Unfortunately the packages didn't work, it keeps saying DLLs not found

After some analysis I can understand a little better how these files work, I mapped the data using ImHex but I have what seems to be the last obstacle to being able to write a functional parser for most files.

only remaining obstacle is the possible pointers to data or strings in the files. It is still unclear whether they are direct pointers, go through an FNV hash, or represent memory addresses. One example is characterAnimationData, which displays a uint32 value of 76241072. These values/addresses/pointers, etc., seem to be located at the end of the file or after the header data. The values appear to be the same, suggesting some kind of relationship between them. In this case, the actual value of characterAnimationData is a string found at the end of the file or after the header. The same pattern occurs with other data entries. However, I still haven't been able to determine what this value (76241072) actually represents, and this happens in most files, most .markerSet file works like this.

I found these messages from dalkon:

 

and that's what I did for the files, but for this type of data that seems to be a key, offset or something like that I still haven't been able to understand how it works, see characterAnimationData:

 

https://drive.google.com/file/d/1FCYNMurfhIX_hXHDvQrUPhgtArvFPuRu/view?usp=drive_link

Edited February 28 by JeanxPereira

[shak-otay]

4 hours ago, JeanxPereira said:

hey! Unfortunately the packages didn't work, it keeps saying DLLs not found

Hey! could you track down to the name(s) of the missing dll(s)?

See here.

(Or, if that doesn't work try this.)

Do you have an IDA screen of HashFunction()?

Edited February 28 by shak-otay

[JeanxPereira]

10 hours ago, shak-otay said:

Hey! could you track down to the name(s) of the missing dll(s)?

here is everything i could capture: these dlls are in System32,
https://imgur.com/a/jUG8eFu

 

i tried running regsvr32 "C:\Windows\System32\api-ms-win-core-synch-l1-2-0.dll":

The module "C:\Windows\System32\api-ms-win-core-synch-11-2-0.dll" was loaded, but the entry point DllRegisterServer was not found.

Make sure "C:\Windows\System32\api-ms-win-core-synch-11-2-0.dll" is a valid DLL or OCX file and try again.

I tried to use another dll with the same name with a larger size but I got the same error

 

10 hours ago, shak-otay said:

Do you have an IDA screen of HashFunction()?

byte_118AB50 and byte_118AC50 appear to be uppercase and lowercase character tables:

 

Edited February 28 by JeanxPereira

[shak-otay]

Did you check the dlls for DLLRegisterServer() inside them?

Quote

If the DLL does not have inside it the DllRegisterServer() (and also DllUnRegisterServer()) functions the regsvr32 program does not know what to do with it - it does not make sense to register it so hence the error message:

the entry-point DllRegisterServer was not found

list exported functions of a dll

 

For the HashFunction(), I'm not sure about the int8 pointer, so we need to check some sources of popular hash functions and try to compare to the asm code you've got.

btw, I don't know whether you know it: you never can get the belonging string out of a hash by using a "reverse formula". It's a one way street from string to hash.

So if you want to know the string belonging to a hash number you'll have to create a table of all strings used in the game and calculate their hashes.

Then searching a hash in that table would bring up the belonging string.

Edited February 28 by shak-otay

[JeanxPereira]

  

4 hours ago, shak-otay said:

Did you check the dlls for DLLRegisterServer() inside them?

no, you are right, using dllexp confirms there is no DLLRegisterServer() inside them

 

4 hours ago, shak-otay said:

For the HashFunction(), I'm not sure about the int8 pointer, so we need to check some sources of popular hash functions and try to compare to the asm code you've got.

it uses FNV Hash with tables byte_118AB50 and byte_118AC50 for lowercase and uppercase 

 

4 hours ago, shak-otay said:

btw, I don't know whether you know it: you never can get the belonging string out of a hash by using a "reverse formula". It's a one way street from string to hash.

So if you want to know the string belonging to a hash number you'll have to create a table of all strings used in the game and calculate their hashes.

Then searching a hash in that table would bring up the belonging string.

Yes, exactly, one of the programs we use to extract the .package files (inside it are the files we are working on and that Testing.exe is capable of extracting) SMFX tool uses this table method to extract the file names correctly, but in this case it seems not to be an FNV Hash that is normally used since this type of key data has similar values (in the case of this file they start with 762...), I created a table to make it easier to understand, opening any noun it is also possible to see that these strings and also some SharedComponentData data are together in sequence.

I converted the same HashFunction logic to a simple python script and tried to use it for these types of offsets/pointers but I couldn't get the same values (offsets or pointers) mentioned in the files, it works for names and other values, I tested it with the SMFX tool own table which has real filenames and their hashes

dataTableNounPointers.zip fnv-hash-calc.py NounFileUsedForExamplesAndPatternFileForIMHEX.zip

Edited February 28 by JeanxPereira