Alan Wake 2

[Contro]

8 hours ago, riverence said:

figured it out 🙂 it also seems as if the vertex size is inconsistent between meshes, which is why i was having trouble importing them. was able to get alan's revolver and his lamp as well

Holy shit, nice! I've been wanting his lamp and revolver for a while now

Edited January 16 by Contro

[riverence]

A new update for the Alan Wake 2 blender addon has been released. This update includes preliminary support for a good chunk of meshes present under /objects/weapons/. I also changed the mesh naming scheme a bit, and streamlined the process for adding any more supported meshes.
If you run into a struct error, the file you selected isn't supported.
 

The newly supported meshes are as follows;

crossbow_default_publish
crossbow_default_publish_physx
doublebarrelshotgun_default_publish
fbiservicepistol_compensator_publish
fbiservicepistol_default_publish
fbiservicepistol_magazine_extended_publish
flaregun_default_publish
flashlight_default_publish
lamp_default_publish
lamp_physx
pumpactionshotgun_compensator_publish
pumpactionshotgun_default_publish_physx
pumpactionshotgun_stock_pouch_publish
revolver_default_publish
rifle_cheek_pad_publish
rifle_default_publish
rifle_default_publish_physx
rifle_scope_publish
sawnoffshotgun_default_publish
sawnoffshotgun_physx

 

io_scene_binfbx.zip

[Mutanttirotta]

Hello,

I am completely new on digging resources from games and I have basically zero experience with coding. I found this thread when I was looking for a way to extract some of the in-game poster files to make me a real poster.

Could you help me and tell is that possible with these tool discussed in this thread and how advanced knowledge would it need to use it for such a task (e.g. is it worth the hustle without any prior knowledge).

Also, do you know if the in-game posters are even in some kind of very high quality scaleable format that it could be used to make some big prints?

I'm sorry that this is hardly related on the matter but figured to ask just in case.

[riverence]

On 1/16/2024 at 5:55 PM, riverence said:

A new update for the Alan Wake 2 blender addon has been released. This update includes preliminary support for a good chunk of meshes present under /objects/weapons/. I also changed the mesh naming scheme a bit, and streamlined the process for adding any more supported meshes.
If you run into a struct error, the file you selected isn't supported.
 

The newly supported meshes are as follows;

crossbow_default_publish
crossbow_default_publish_physx
doublebarrelshotgun_default_publish
fbiservicepistol_compensator_publish
fbiservicepistol_default_publish
fbiservicepistol_magazine_extended_publish
flaregun_default_publish
flashlight_default_publish
lamp_default_publish
lamp_physx
pumpactionshotgun_compensator_publish
pumpactionshotgun_default_publish_physx
pumpactionshotgun_stock_pouch_publish
revolver_default_publish
rifle_cheek_pad_publish
rifle_default_publish
rifle_default_publish_physx
rifle_scope_publish
sawnoffshotgun_default_publish
sawnoffshotgun_physx

 

io_scene_binfbx.zip 10.01 kB · 15 downloads

Currently working on getting the rig out of the game, with some great progress made so far. 1; I was able to implement the boundary box for each submesh correctly, so, woohoo, and 2; I'm able to get the actual skeleton out, posed correctly and everything. After I get the weights out and get them applied to the mesh, I'll be working on possibly writing mesh data to an existing *.binfbx file (because creating new ones would probably be hell) as well as implementing a universal solution for the binfbx format that doesn't require manually finding the offsets. Once I have more news, I'll share it here, but in the meantime if anyone here is specifically proficient with Blender's Python API, and knows at least a little bit of the format, I'd love some help if possible.

[riverence]

 

Update 2.00 - Alan Wake 2 *.binFBX Blender Addon

• added rigged skeleton support, including non-human meshes (let me know if you get anything unexpected, still rough around the edges!)
• added support for some other meshes i can't remember
• more debug logs for anyone interested in the format
• died and came back to life

Roadmap

• material generation
• proper reverse engineering of mesh file (every mesh file being supported without manually finding offsets)
• mesh/submesh replacing (might be coming the soonest)

io_scene_binfbx.zip

[Kanbara]

5 hours ago, riverence said:

 

Update 2.00 - Alan Wake 2 *.binFBX Blender Addon

You rock!!!!!!!!!!😍😚

[Mike2023]

@riverence

Wow, you put a lot of work into the script, my respect for you! Recently found the Blender Import Script from Control, just have a look at, regarding the offsets, shaders, textures, etc. 😉

[riverence]

On 2/5/2024 at 11:15 PM, riverence said:

 

Update 2.00 - Alan Wake 2 *.binFBX Blender Addon

• added rigged skeleton support, including non-human meshes (let me know if you get anything unexpected, still rough around the edges!)
• added support for some other meshes i can't remember
• more debug logs for anyone interested in the format
• died and came back to life

Roadmap

• material generation
• proper reverse engineering of mesh file (every mesh file being supported without manually finding offsets)
• mesh/submesh replacing (might be coming the soonest)

io_scene_binfbx.zip 9.19 kB · 31 downloads

I'll also be updating it on GitHub, feel free to contribute or to open issues there (preferred); https://github.com/riverence/io_scene_binfbx

Edited February 18 by riverence

[bladers]

On 2/18/2024 at 12:12 PM, riverence said:

I'll also be updating it on GitHub, feel free to contribute or to open issues there (preferred); https://github.com/riverence/io_scene_binfbx

Are there documentation or tutorial on how to add support for a mesh (particularly a none skeletal mesh)?

I could even add code to try to automate the process.

[bladers]

On 2/2/2024 at 7:02 AM, riverence said:

as well as implementing a universal solution for the binfbx format that doesn't require manually finding the offsets. Once I have more news, I'll share it here, but in the meantime if anyone here is specifically proficient with Blender's Python API, and knows at least a little bit of the format, I'd love some help if possible.

From what I understand...

The table_offset variable is used to specify the starting point within the .binFBX file from which the script begins to read mesh information? The correct value for table_offset depends on the specific .binFBX file being imported, as different files may have their data structured differently. Different assets have their mesh data tables starting at different points within the files.

The lod0_offset variable indicates the starting point within the .binFBX file for the mesh data corresponding to the Level of Detail 0 (LOD0). The offset is used to locate where the vertex and face data for this LOD begin in the file.

So the question now is, how are you inspecting the files to get the table offset and lod0 offset? Are you just opening them with notepad or some debugger tool? Is there an example?

[riverence]

11 hours ago, bladers said:

From what I understand...

The table_offset variable is used to specify the starting point within the .binFBX file from which the script begins to read mesh information? The correct value for table_offset depends on the specific .binFBX file being imported, as different files may have their data structured differently. Different assets have their mesh data tables starting at different points within the files.

The lod0_offset variable indicates the starting point within the .binFBX file for the mesh data corresponding to the Level of Detail 0 (LOD0). The offset is used to locate where the vertex and face data for this LOD begin in the file.

So the question now is, how are you inspecting the files to get the table offset and lod0 offset? Are you just opening them with notepad or some debugger tool? Is there an example?

Yes, the data table offset is different between files, as the skeleton info at the beginning of the file is of a variable size, and there's sometimes some mystery data after that, and only then does the mesh data table actually begin.

What I've been doing to find it reliably is to open up a mesh file in ImHex, and search for the binary sequence of '01 07 02 00', which seems to be used as the beginning of a separator between submeshes in the data table. The number of times this separator appears is typically the amount of submeshes, and you can use that number to find the Mesh ID specified in the Blender addon. You may catch some strays of the sequence in other random mystery data, but in that case just find where the first instance of it occuring in the table starts and add the next 4 bytes to the search, as they are part of the same separator, just per-mesh type. It's only typical for the non 32-byte vertex meshes, from my experience.

After this, finding the lod0 offset is a bit trickier but typically you'll be able to find it by navigating to the vertex offset (you can find out more about the pattern per size if looking for it manually by checking the offsets in supported files output by Blender), taking the number of vertices and multiplying it by 8 to get the number of bytes of the UV table, which you can then use (as a negative number in ImHex's Ctrl-F > Relative tab, just make sure you've selected the exact byte the vertices start at) to navigate relatively from the vertex offset to the lod0 offset.

This process has generally worked for me, but it's incredibly finicky and sometimes won't work in mysterious cases (cough cough, Alan's typewriter).

Some meshes also require finding a specific offset for the skeleton located in the mesh, which is an entire other can of worms as Remedy has seemingly attached a completely different file to the beginning of some mesh files, thus displacing the start of the actual mesh file by a not-insignificant number of bytes. This is not the case for all files, and it's currently unknown what the appended data is actually doing.

This is all I know so far, let me know if you have any questions, I know some of my wording is confusing.

Edited February 25 by riverence

[Mike2023]

Edit: It looks like the script only works for the files where the offset are known right! Sorry guys, still have to put a little work on the script! 🤔

Edit: File deleted. Please look for the last post of the search file.

 

Edited May 18 by Mike2023

[riverence]

17 hours ago, Mike2023 said:

Edit: It looks like the script only works for the files where the offset are known right! Sorry guys, still have to put a little work on the script! 🤔

AW2_search_offsets_us.py 3 kB · 4 downloads

Hey hey! While checking it out, I noticed that while the first 4 offsets given (for a mesh with 6 lods from 5-0) seem to be correct, the last one isn't and typically points to the end of the file or some other unknown offset. It also seems to ignore the first LOD offset, that being the LOD that begins right after the data table ends. Other than that, thank you for your work! I've begun integrating it into the Blender addon, but with not much luck. You can check out my progress on the most recent commit at https://github.com/riverence/io_scene_binfbx/blob/main/__init__.py

[riverence]

5 hours ago, riverence said:

Hey hey! While checking it out, I noticed that while the first 4 offsets given (for a mesh with 6 lods from 5-0) seem to be correct, the last one isn't and typically points to the end of the file or some other unknown offset. It also seems to ignore the first LOD offset, that being the LOD that begins right after the data table ends. Other than that, thank you for your work! I've begun integrating it into the Blender addon, but with not much luck. You can check out my progress on the most recent commit at https://github.com/riverence/io_scene_binfbx/blob/main/__init__.py

It seems like the problem here is that the method for finding the LOD offset is unreliable, especially in non-human meshes, as the bytes searched for right before the beginning of the LOD aren't the same. I'll look into alternatives right now, or if you can find one, keep me updated 😄

[Mike2023]

On 2/27/2024 at 8:20 PM, riverence said:

It seems like the problem here is that the method for finding the LOD offset is unreliable, especially in non-human meshes, as the bytes searched for right before the beginning of the LOD aren't the same. I'll look into alternatives right now, or if you can find one, keep me updated 😄

Table_offset is found correctly, but there are different block sizes with the same MAGIC_SIG. The standard script uses the block size of 127 bytes from cur_offset. If you now set this value to 119, the files will be read correctly by the table_offset.
LOD_0 also has a different padding of 4, instead of 26 as in the standard script.

Edit: I just saw it! block_offset = 119 in your Script! 😄

Edited February 28 by Mike2023

[riverence]

47 minutes ago, Mike2023 said:

Table_offset is found correctly, but there are different block sizes with the same MAGIC_SIG. The standard script uses the block size of 127 bytes from cur_offset. If you now set this value to 119, the files will be read correctly by the table_offset.
LOD_0 also has a different padding of 4, instead of 26 as in the standard script.

Edit: I just saw it! block_offset = 119 in your Script! 😄

Yeah, the block_offset changes depending on the vertex size. You can find the vertex size by looking at the int32 for the number of bones per vertex; 8 = 32 byte vertex and a 127 byte block offset, 4 = 16 byte vertex and a 123 byte block offset, 1 = 12 byte vertex and a 119 byte block offset, and if it's 0 it's an 8 byte vertex with a 115 byte block offset. You can find the bonesPerVertex int32 right after the face_offset int32. It may be something similar when it comes to the LOD0 offsets, but I'm not totally sure. It also seems like it's possible to navigate to the next LOD offset by simply navigating through the UV table, vertex table, and the face table by calculations, but there is a table or two after the face table that's completely unknown to us right now. However, after that table, the next LOD offset immediately begins.

[Mike2023]

On 2/28/2024 at 11:37 PM, riverence said:

Yeah, the block_offset changes depending on the vertex size. You can find the vertex size by looking at the int32 for the number of bones per vertex; 8 = 32 byte vertex and a 127 byte block offset, 4 = 16 byte vertex and a 123 byte block offset, 1 = 12 byte vertex and a 119 byte block offset, and if it's 0 it's an 8 byte vertex with a 115 byte block offset. You can find the bonesPerVertex int32 right after the face_offset int32. It may be something similar when it comes to the LOD0 offsets, but I'm not totally sure. It also seems like it's possible to navigate to the next LOD offset by simply navigating through the UV table, vertex table, and the face table by calculations, but there is a table or two after the face table that's completely unknown to us right now. However, after that table, the next LOD offset immediately begins.

Thank you for the explanation and sorry for getting in touch so late.
Here I have something new for you to test. In addition, the manual offsets were stored in a json file. I also have to put a little more time into calculating for the total LOD offsets of the files. Have a nice weekend everyone!

Edit: I have a revised version of the script here. Unfortunately, it's still not possible without the Hex Editor, but it's a great help in the search.

@riverence

For some meshes (e.g. npc_ <- at the beginning), your script must load the vertex data with 20 bytes (vertex size: 16) instead of 32 bytes.

Edit: File deleted. Please look for the last post of the search file.

 

Edited May 18 by Mike2023

[nexus2641]

Hi guys. This is the error that blender gives when importing any model.

 

Python: Traceback (most recent call last):
  File "C:\Users\nexus\AppData\Roaming\Blender Foundation\Blender\3.5\scripts\addons\io_scene_binfbx\__init__.py", line 1151, in execute
    return import_binfbx(context, self.filepath, self.import_rig, self.file_structure)
  File "C:\Users\nexus\AppData\Roaming\Blender Foundation\Blender\3.5\scripts\addons\io_scene_binfbx\__init__.py", line 1112, in import_binfbx
    main()
  File "C:\Users\nexus\AppData\Roaming\Blender Foundation\Blender\3.5\scripts\addons\io_scene_binfbx\__init__.py", line 1105, in main
    read_mesh_skel(skel, file, skel_info, collection_name, objArray)
  File "C:\Users\nexus\AppData\Roaming\Blender Foundation\Blender\3.5\scripts\addons\io_scene_binfbx\__init__.py", line 695, in read_mesh_skel
    bpy.data.scenes['Scene'].collection.objects.unlink(arm_obj)
RuntimeError: Error: Object "Armature" is not in the "Scene Collection"

[nexus2641]

This error occurs when importing "kiran_estevez_healed_publish"

Python: Traceback (most recent call last):
  File "C:\Users\nexus\AppData\Roaming\Blender Foundation\Blender\3.6\scripts\addons\io_scene_binfbx\__init__.py", line 1151, in execute
    return import_binfbx(context, self.filepath, self.import_rig, self.file_structure)
  File "C:\Users\nexus\AppData\Roaming\Blender Foundation\Blender\3.6\scripts\addons\io_scene_binfbx\__init__.py", line 1112, in import_binfbx
    main()
  File "C:\Users\nexus\AppData\Roaming\Blender Foundation\Blender\3.6\scripts\addons\io_scene_binfbx\__init__.py", line 1052, in main
    vertArray = read_mesh_vertices(file, mesh_info, skel_info, vertex_offset, minv, maxv, i - 1)
  File "C:\Users\nexus\AppData\Roaming\Blender Foundation\Blender\3.6\scripts\addons\io_scene_binfbx\__init__.py", line 818, in read_mesh_vertices
    if xMin > lva[vi][0]:
IndexError: list index out of range
 

[riverence]

15 hours ago, nexus2641 said:

Hi guys. This is the error that blender gives when importing any model.

 

Python: Traceback (most recent call last):
  File "C:\Users\nexus\AppData\Roaming\Blender Foundation\Blender\3.5\scripts\addons\io_scene_binfbx\__init__.py", line 1151, in execute
    return import_binfbx(context, self.filepath, self.import_rig, self.file_structure)
  File "C:\Users\nexus\AppData\Roaming\Blender Foundation\Blender\3.5\scripts\addons\io_scene_binfbx\__init__.py", line 1112, in import_binfbx
    main()
  File "C:\Users\nexus\AppData\Roaming\Blender Foundation\Blender\3.5\scripts\addons\io_scene_binfbx\__init__.py", line 1105, in main
    read_mesh_skel(skel, file, skel_info, collection_name, objArray)
  File "C:\Users\nexus\AppData\Roaming\Blender Foundation\Blender\3.5\scripts\addons\io_scene_binfbx\__init__.py", line 695, in read_mesh_skel
    bpy.data.scenes['Scene'].collection.objects.unlink(arm_obj)
RuntimeError: Error: Object "Armature" is not in the "Scene Collection"

Remove the previous collection, and make sure that the topmost Scene Collection is selected when importing meshes. As for the 'list index out of range' issue, I'm still trying to figure out how to fix that.

[nexus2641]

17 hours ago, riverence said:

Remove the previous collection, and make sure that the topmost Scene Collection is selected when importing meshes. As for the 'list index out of range' issue, I'm still trying to figure out how to fix that.

Thanks for the answer.

[Mike2023]

The files are really very complex and sometimes even built in layers. That is why a dynamic offset calculation of the LODS is necessary, as I will probably pair it to the byte sequence search. But here is a new version of the script with layer support and other improvements. 🙂

Edit: File deleted. Please look for the last post of the search file.

 

Edited May 18 by Mike2023

[DKDave]

Not sure if this is much use as there's still a lot of unknowns in the mesh files, but I've been looking at them for a while as well.

The block of unknown data after the faces starts with a bunch of tables containing offsets to some data.  The counts and offsets for these tables are in the LOD mesh entries just before the final table (if it exists).  Then each of these unknown tables has a bunch of offsets to some more data, then the next table follows.  After that there's yet more unknown data that looks like it's referenced from those submeshes that have the extra table in them, with some more offsets.  Then after that is the data for the next LOD.  But it's still a bit uncertain how to combine all that info to calculate everything.

 

 

[Mike2023]

On 3/10/2024 at 12:14 AM, DKDave said:

Not sure if this is much use as there's still a lot of unknowns in the mesh files, but I've been looking at them for a while as well.

The block of unknown data after the faces starts with a bunch of tables containing offsets to some data.  The counts and offsets for these tables are in the LOD mesh entries just before the final table (if it exists).  Then each of these unknown tables has a bunch of offsets to some more data, then the next table follows.  After that there's yet more unknown data that looks like it's referenced from those submeshes that have the extra table in them, with some more offsets.  Then after that is the data for the next LOD.  But it's still a bit uncertain how to combine all that info to calculate everything.

 

 

The files are so complex that some UV sets for a Lod start in the middle of a data block. Nevertheless, I tried to incorporate automation into the script, which supports different files, not only the characters.

An error message occurs to the script run if there are no offsets in the JSON for calculating. Then activate the Lod0_offset and run the script again. Then enter one of the offsets found at Lod0_offset and run the script again to carry out the calculations of the UV, Vertex and Face-Offsets.

I still write Parrallel on a script to correctly integrate the shaders from the material files. I have never had much to do with Shader, Tiling and Co in the past!😅

Here would be the next version of the search script for testing...

Edit: File deleted. Please look for the last post of the search file.

 

Edited May 18 by Mike2023

[Mike2023]

I redid the script logic, it took a little brain power... 🫡 
The script now runs somewhat, you can leave it like that for now. Have fun with it! 😊

AW2_search_offsets_us_v3.zip