Models in EA rsf files

[lorak]

A bit more than one year ago, back in XeNTaX forums, I was asking for help to get models form EA rsf files via hex2obj, as I found a few related threads over there. I got a few useful replies from @shak-otay. Unfortunately, XeNTaX went down. Now, my interest in editing these kind of files just restarted.

The attached file has a sample rsf file from Madden 13 PS Vita with the model of the Miami Dolphins stadium in different sub meshes. From those old threads at XenTaX, I know how to identify vertices counts, FVF size and coordinates for each vertex as those appear after the word "MRTS" for each sub-mesh in these rsf files. Similarly, the face indices appear after the word "XDNI" for each sub-mesh.

There are other EA rsf files with the same number of "XDNI" and "MRTS" blocks. Thus, the correspondence between them is easy to find, as we have for example, six blocks for each word ("MRTS" and "XDNI") meaning six sub-meshes. Vertices information and FVF size for the first sub-mesh are in the first "MRTS" block and faces indices are in the first "XDNI" block and so on. However, in the sample file I found 120 times the word "XDNI" and 287 times the word "MRTS". Now, I have a couple of questions:

1. I saved your replies from XeNTaX, and know that this PS Vita file has 120 or 121 sub-meshes (maybe linked to the number of faces indices blocks?). How to identify that number as the number of blocks with the words "MRTS" and "XDNI" are different?

2. How to find the uv?

By the way, the sample file for PS Vita is in little endian. There are very similar rsf files for other versions of Madden for PS3 with more recent stadium updates. The main difference is the data endianess, little endian for PS Vita and big endian for PS3. For completeness, I also have included the Miami Dolphins stadium rsf file from Madden 25 PS3.

DOLPHINS Stadium -M13 PS Vita & M25 PS3-.zip

Edited December 3 by lorak

[shak-otay]

5 hours ago, lorak said:

1. I saved your replies from XeNTaX, and know that this PS Vita file has 120 or 121 sub-meshes (maybe linked to the number of faces indices blocks?). How to identify that number as the number of blocks with the words "MRTS" and "XDNI" are different?

2. How to find the uv?

I don't remember anything to be honest.

Maybe you could give a short overview about the old informations?

 

[lorak]

17 hours ago, shak-otay said:

I don't remember anything to be honest. Maybe you could give a short overview about the old informations?

Perhaps shakotay2 from XeNTaX is not you, no idea.
Anyway, I share these screenshots from the old threads at XeNTaX where shakotay2 share some information about the Dolphins stadium in the PS Vita rsf file attached to my previous post.

Very similar rsf model files, appear in the game Nascar 08 for PS2 made by EA Sports. There was a thread about that in XeNTaX too.

Edited December 3 by lorak

[shak-otay]

9 hours ago, lorak said:

Perhaps shakotay2 from XeNTaX is not you, no idea.

You can never be sure - but looks familiar;-)

Do you have one of the models handled in the screenshots?

Since the need of a script was mentioned I might try out hscript on .RSF (once it's complete).

 

[lorak]

6 hours ago, shak-otay said:

You can never be sure - but looks familiar;-)

I agree. 😉

6 hours ago, shak-otay said:

Do you have one of the models handled in the screenshots?

The obj files (model and UVs) of sub-mesh 38 are in the attachment. I am not 100% sure if this is what you want.

6 hours ago, shak-otay said:

Since the need of a script was mentioned I might try out hscript on .RSF (once it's complete).

It seems to be great news to extract these models in a simpler way!!
Are you talking about hscript found here or here?

Dolphins-stadium-sub_mesh_38.zip

Edited December 3 by lorak

[shak-otay]

3 hours ago, lorak said:

Are you talking about hscript found here or here?

Well, no. hscript is very simple with a few commands only (think of them as "macros"):

here's the reduced commands list:

endian 1	// switches to big endian
endian 0	// switches to little endian
		// little endian is default, btw
skip  [count]   // skip count bytes

start [address] // address in hex always, WITHOUT the 0x
loop [count]	// perfoms the commands in the loop block count times
loop 0		// end of loop block

rbyte [count]	// reads count bytes
rfloat [count]  // reads count floats
rhfloat [count] // reads count half floats

rstr 0		// reads zero terminated string
rstr 333        // read string with preceeding byte counter
rstr 444        // read string with preceeding word counter
rstr 555        // read string with preceeding DWORD counter

rstr [count]    // just for completeness

The functions the commands use are those of hex2obj (or the make_obj project).

The script parser I need is very simple, too, because its only ability is to read string/number pairs in a loop.

Something like fscanf(stream, "%s %d\n", szCmd, &count);

The problem to solve: how to do a loop in a macro interpreter?

It's just an adding of [count] command blocks to the final commands sequence but I'm a little bit stuck at this point.

If we had to read 10000 x 3 floats of a vertex block it would sum up to 10000  lines, i.e. the parser

would expand

loop 10000

rfloat 3

loop 0

to 10000 lines

rfloat 3

which would slow down the whole thing.

edit: well, I'm not the guy to solve such theoretically. I need to go into practice.

Edited December 3 by shak-otay

[lorak]

11 minutes ago, shak-otay said:

The problem to solve: how to do a loop in a macro interpreter?

It's just an adding of [count] command blocks to the final comands sequence but I'm a little bit stuck at this point.

If we had to read 10000 x 3 floats of a vertex blocks it would sum up to 10000  lines, i.e. the parser

would expand

loop 10000

rfloat 3

loop 0

to 10000 lines

rfloat 3

which would slow down the whole thing.

Could you run the loop outside of the macro interpreter via a master script or so?

[shak-otay]

There's many solutions, I think. But it should be kept as simple as possible.

I reduced the problem to "how to pass parameters from hscript to c-function level".

In the source of Make_obj (see tools forum, iirc) you find this:

void Vertex12(FILE *stream, DWORD dwStart, DWORD dwVertsCnt, BYTE vStride, bool bBigE)

So it's not required to loop on macro level, of course.

Next problem: the parser needs to do some "translation" for

loop 10000

rfloat 3

skip 20

loop 0

such as vStride= 12+20

Seems I'll need some kind of intermediate level just for vStride.

That's not nice.

(Well, I don't want to spam this thread with unrelated things.)

Hopefully I can release hscript soon.

Good night!

edit: all possible problems described before have dissolved into thin air, haha.

Most features are already present in hex2obj.

"Getting multiple submeshes by script" is the task to be solved.

And it might be specific/different to every 3D format.

(Next post shows an example what has to be done.)

Edited December 4 by shak-otay

[shak-otay]

16 hours ago, lorak said:

It seems to be great news to extract these models in a simpler way!!

I thought so but

since you still need to have the skills to find the first (or one) sub mesh (i.e. know floats and integers and how vertices, normals, uvs  and face indices are arranged in blocks)
the correct phrasing would be: "hscript will make finding sub meshes a little bit easier".

Anyways here's the next sub mesh:

I have no idea why it should be 121 sub meshes; I'd guess for 127. Anyways, here's some calculations required to get the start addresses

after you've searched for MRTS (vertex blocks) and XDNI (face index blocks):

First face indices (FIs) block, FI count:
1cba6d - 1cb02a = 0xa43= 2627 dec. -> 1313 words (face indices)

2nd FIs block, FI count:
1cc16e - 1cba6d = 0x701 -> 1793 dec. -> 896 words (face indices)

add 0x11 to 0x1cba6d -> 0x1cba7e
0x11= 17 dec. is 8 words which need to be subtracted of 896, so 888 face indices remain

Enter address and count near the go1 button, press it, scroll down in left lower combo box you'll find 200 as vertex count (== max face index).

You can compare that value to the DWORD 9 bytes after the MRTS signature.

Finding the suiting vertex block address for the first FIs block is tricky. Using 0x21c9 (0x21b4 + 0x15) will result in bogus.
You'll need to assure that it's a floats vertex block by checking the byte at offset 0x0D (-> address 0x21c1) for 0x0C (12 bytes for a vertex float). It's 0x14, so not what we need.

Next fitting address is 0x6a37 (+ 0x0D= 0x6a44, contains 0x0C).
So vertex start at 0x6a37 + 0x15= 0x6a4c

This is what hscript should process but it's the user who needs to provide the suiting commands. I have my doubts.

And before having coded a single parser line I found it has to be revised. It's basic features are already present in hex2obj.

What's missing is "looping sub meshes". Finding signatures could help a lot with it.

find [signature]

We also need skip [count] again. Here count is 0x15 for the vertex signature and 0x11 for XDNI.

 

 

Edited December 4 by shak-otay

[lorak]

7 hours ago, shak-otay said:

Since you still need to have the skills to find the first (or one) sub mesh (i.e. know floats and integers and how vertices, normals, uvs  and face indices are arranged in blocks)
the correct phrasing would be: "hscript will make finding sub meshes a little bit easier".

I have no idea why it should be 121 sub meshes; I'd guess for 127. Anyways, here's some calculations required to get the start addresses

after you've searched for MRTS (vertex blocks) and XDNI (face index blocks):

First face indices (FIs) block, FI count:
1cba6d - 1cb02a = 0xa43= 2627 dec. -> 1313 words (face indices)

2nd FIs block, FI count:
1cc16e - 1cba6d = 0x701 -> 1793 dec. -> 896 words (face indices)

add 0x11 to 0x1cba6d -> 0x1cba7e
0x11= 17 dec. is 8 words which need to be subtracted of 896, so 888 face indices remain

Enter address and count near the go1 button, press it, scroll down in left lower combo box you'll find 200 as vertex count (== max face index).

You can compare that value to the DWORD 9 bytes after the MRTS signature.

I understand your calculations. For this kind of rsf files, the faces indices appear 9 bytes after the word XDNI for each sub-mesh as shown in the highlighted data in the first screenshot for the first and second sub-meshes. It matches your computations.

 

7 hours ago, shak-otay said:

Finding the suiting vertex block address for the first FIs block is tricky. Using 0x21c9 (0x21b4 + 0x15) will result in bogus.
You'll need to assure that it's a floats vertex block by checking the byte at offset 0x0D (-> address 0x21c1) for 0x0C (12 bytes for a vertex float). It's 0x14, so not what we need.

Next fitting address is 0x6a37 (+ 0x0D= 0x6a44, contains 0x0C).
So vertex start at 0x6a37 + 0x15= 0x6a4c

This is what hscript should process but it's the user who needs to provide the suiting commands. I have my doubts.

And before having coded a single parser line I found it has to be revised. It's basic features are already present in hex2obj.

What's missing is "looping sub meshes". Finding signatures could help a lot with it.

find [signature]

We also need skip [count] again. Here count is 0x15 for the vertex signature and 0x11 for XDNI.

Yes, this is tricky. I was not able to easily find the vertex block (MRTS) associated to a faces indices block (XDNI).
Ok, if I correctly understand, to find the vertex block associated with a face indices block we need to look for an MRTS block where the number of vertices matches what we see in hex2obj (max face index through the go1 button) 9 bytes after the word MRTS AND having number 0x0C D bytes after the word MRTS.

If this is the case, why the MRTS block starting at offset 0x510D is not suitable as vertex block for the second sub-mesh?
This MRTS block fulfills all those requirements (200 vertex count and 0x0C value D bytes after the word MRTS) Check the second screenshot.

Many thanks for the insight!

Edited December 4 by lorak

[lorak]

@shak-otay What do you do to show the faces in the model in hex2obj? The blue lines on the model.

[shak-otay]

2 hours ago, lorak said:

If this is the case, why the MRTS block starting at offset 0x510D is not suitable as vertex block for the second sub-mesh?

Upps, seems I skipped that...

Quote

@shak-otay What do you do to show the faces in the model in hex2obj? The blue lines on the model.

Just press 'e'.

[lorak]

Following your explanations above we can find the vertices block (MRTS) associated to a given faces indices block (XDNI). It seems to me that the next MRTS block corresponds to the normals and the following one corresponds to the UVs. Do you agree @shak-otay? 

[shak-otay]

7 hours ago, lorak said:

Following your explanations above we can find the vertices block (MRTS) associated to a given faces indices block (XDNI). It seems to me that the next MRTS block corresponds to the normals and the following one corresponds to the UVs. Do you agree @shak-otay? 

Sounds good - but if we assume the existence of about 120 sub meshes that would mean 3x120 MRTS blocks while it's 287 only.

edit: point cloud of Dolphins Madden 13:

Edited December 5 by shak-otay

[shak-otay]

The following is obsolete! Improved correction (see next post).

(Instead of erasing this I leave it to give interested an overview.)

Here's the (uncomplete) result (39 wrong objx files).

Some vBlock - FIblock combinations were fixed, 39 left unsolved because of not fitting vCnt (vBlocks) and maxFIcnt (FIblocks).

Here's the list of faulty sub meshes (FI block numbers):

Some corrections failed because of too big block number offsets (108 versa 120):

FIblocks
108 vCnt 1049 != 16  (FIcnt 1656)
109 vCnt 1110 != 360  (FIcnt 1752)
110 vCnt 1293 != 135  (FIcnt 2145)

should be "connected" to

vBlocks
120 1b3e3f 1049
121 1ba0c1 1110
122 1c08fb 1293

(which will give the spectator stands)

correction failed comparison maxFIcnt to vCnt
=================
FI block number in first column

6 vCnt 266 != 299  (FIcnt 612)
7 vCnt 8 != 2262  (FIcnt 12)
8 vCnt 8 != 1352  (FIcnt 12)
9 vCnt 4 != 32  (FIcnt 6)

82 vCnt 231 != 12  (FIcnt 360)
83 vCnt 74 != 2362  (FIcnt 108)
84 vCnt 484 != 12  (FIcnt 732)
85 vCnt 60 != 231  (FIcnt 114)
86 vCnt 30 != 3  (FIcnt 120)
87 vCnt 22 != 74  (FIcnt 36)
88 vCnt 148 != 484  (FIcnt 258)
89 vCnt 186 != 60  (FIcnt 315)
90 vCnt 70 != 4  (FIcnt 120)
91 vCnt 442 != 30  (FIcnt 714)
92 vCnt 88 != 22  (FIcnt 156)
93 vCnt 596 != 148  (FIcnt 1032)
94 vCnt 625 != 186  (FIcnt 1920)
95 vCnt 68 != 70  (FIcnt 126)
96 vCnt 808 != 442  (FIcnt 1212)
97 vCnt 318 != 88  (FIcnt 726)
98 vCnt 1856 != 596  (FIcnt 2784)
99 vCnt 16 != 4  (FIcnt 24)
100 vCnt 360 != 625  (FIcnt 1536)
101 vCnt 135 != 4  (FIcnt 576)
102 vCnt 45 != 4  (FIcnt 192)
103 vCnt 225 != 68  (FIcnt 960)
104 vCnt 96 != 808  (FIcnt 144)
105 vCnt 304 != 318  (FIcnt 456)
106 vCnt 164 != 4  (FIcnt 246)
107 vCnt 593 != 1856  (FIcnt 897)
108 vCnt 1049 != 16  (FIcnt 1656)
109 vCnt 1110 != 360  (FIcnt 1752)
110 vCnt 1293 != 135  (FIcnt 2145)
111 vCnt 10 != 45  (FIcnt 18)
112 vCnt 195 != 45  (FIcnt 297)
113 vCnt 10 != 225  (FIcnt 18)
114 vCnt 194 != 4  (FIcnt 294)
115 vCnt 26 != 4  (FIcnt 42)
116 vCnt 16 != 96  (FIcnt 24)

And a 2nd list with objx files and the numbers of H2O files they have been created from.

Has been fixed (see next post) :

objx files and their creator H2O file numbers
=============================================
objx 99 -> 83 H2o

101 85

103 87
104 88
105 89

106 9
107 90
108 91
109 92
//100 84  bogus

10 107
11 108
12 109
14 110

objx 15 H20 111 ...objx 20 H2o 116

110 93 ...116 99

3 100 ...7 104
8 105
9 106

73 6
84 7
95 8

98 82

That wasn't a matter of hscript, of course - just some preparations. But I don't think the corrections could be made using hscript.

(I used the Make_H2O project for it.)

 

 

Edited December 5 by shak-otay
outdated information

[shak-otay]

Unzip appended zipped H2O files.

Load Dolphins Madden 13 PC Vita.rsf into hex2obj, then choose File\SaveAs Mmesh

unresolved:
73 objx H2O 6  vCnt 299 versa maxFI 266 see 5_H2O
94 objx H2O 7  vCnt 2262 versa maxFI 8 ??
95 objx H2O 8  vCnt 1352 versa maxFI 8 see 10_H2O

Python script to import multiple obj files. Create a text window in blender, copy code into it, press alt-p to execute:

(path adjustment required!)

import os
import bpy

# http://blender.stackexchange.com/questions/5064/batch-import-wavefront-obj
# put the location to the folder where the objs are located here in this fashion
# this line will only work on windows ie C:\objects
path_to_obj_dir = os.path.join('D:\\', 'test\\objects')

# get list of all files in directory
file_list = sorted(os.listdir(path_to_obj_dir))

# get a list of files ending in 'obj'
obj_list = [item for item in file_list if item[-3:] == 'obj']

# loop through the strings in obj_list and add the files to the scene
for item in obj_list:
    path_to_file = os.path.join(path_to_obj_dir, item)
    bpy.ops.import_scene.obj(filepath = path_to_file)

edit: uvs are half floats, example:

0x1EEDE0 1650
Vb1
12 99
0x10A1F3 928
020100
0x10CD90 12

Fix needed: about half of the H2Os with uv have a wrong uv addr.

edit2: 22 of them fixed. 8 remaining H2Os (_0.H2O to _7.H2O) have an uv addr 0x0.

 

DOLPHINS-Madden 13 PS Vita.RSF_H2O.zip

 

DOLPHINS-Madden 13 PS Vita.RSF_H2O-uvs.zip

Edited December 6 by shak-otay
updated H2O uvs

[lorak]

6 hours ago, shak-otay said:

Unzip appended zipped H2O files.

Load Dolphins Madden 13 PC Vita.rsf into hex2obj, then choose File\SaveAs Mmesh

unresolved:
73 objx H2O 6  vCnt 299 versa maxFI 266 see 5_H2O
94 objx H2O 7  vCnt 2262 versa maxFI 8 ??
95 objx H2O 8  vCnt 1352 versa maxFI 8 see 10_H2O

Python script to import multiple obj files. Create a text window in blender, copy code into it, press alt-p to execute:

The model looks amazing!! 🤩
I assume you really figured out all the model data in the rsf file besides just one block (94 objx).

Is the script you shared above equal to "load_multi_obj.py" mentioned in hex2obj?
Looking at those h2o files, its data is just what we need to put as input in hex2obj to render each sub-mesh. Probably the script you were talking about the last two days is intended to create all those h2o files automatically.

What is a correction? Artificially matching the vertex count and the maximum faces indices count?

6 hours ago, shak-otay said:

edit: uvs are half floats, example:

0x1EEDE0 1650
Vb1
12 99
0x10A1F3 928
020100
0x10CD90 12

Fix needed: about half of the H2Os with uv have a wrong uv addr.

Yes, the uv addresses are difficult to find.

Edited December 5 by lorak

[shak-otay]

36 minutes ago, lorak said:

Is the script you shared above equal to "load_multi_obj.py" mentioned in hex2obj?

Yep. Besides the path.

Quote

Probably the script you were talking about the last two days is intended to create all those h2o files automatically.

As I wrote I used the Make_H2O project ("C" language). Although the rsf mesh format is rather simple hscript is not useful here because the MRTS and XDNI blocks are not synchronous.

(Some of them contain uv data and I didn't find an uv block for each sub mesh.)

Quote

What is a correction? Artificially matching the vertex count and the maximum faces indices count?

It works like so: for each face index block the max face index is calculated (which usually is identical to the vertex count of a suiting v block.)

So there's a list of MRTS block numbers (plus counts) and one for the XDNI numbers (plus counts).

Now all blocks have to be compared concerning identical counts to combine matching vertex and faceIndex blocks.

(Maybe there's a simpler solution but that's how I did it.) Good night...

[lorak]

7 hours ago, shak-otay said:

As I wrote I used the Make_H2O project ("C" language). Although the rsf mesh format is rather simple hscript is not useful here because the MRTS and XDNI blocks are not synchronous.

I made a quick search, could not find Make_H2O project. Where can I find it?
I just found another thread here in reshax from last year with a Make_H2O-SF.exe. It seems the tool I found in that thread cannot load the Dolphins stadium rsf file.

[shak-otay]

Here's Make_obj source. It includes most functions which Make_H2O has (except for Create_H2Os()).

(It might only help you if you're a coder.)

(Make_H2O.exe for RSF will be released as soon as it's confirmed to work with another sample.)

Edited December 6 by shak-otay
bolding obj

[Durik256]

I have already made a plugin for NHL 21 (PS4) and NASCAR 08 (PS3) (*rsf)

*(and if I remember correctly and they all have different MESH blocks, yours is no exception. regarding UVS, vertices and UVS can be in different STRMs, this is indicated in the MESH blocks)

[lorak]

7 hours ago, Durik256 said:

I have already made a plugin for NHL 21 (PS4) and NASCAR 08 (PS3) (*rsf)

*(and if I remember correctly and they all have different MESH blocks, yours is no exception. regarding UVS, vertices and UVS can be in different STRMs, this is indicated in the MESH blocks)

Yeah, I remember in XeNTaX a Noesis plugin to handle rsf files, most likely it appeared in your posts there.

I just searched for HSEM (MESH) blocks and found 157. It seems the length of those HSEM blocks is 0x30 bytes. I highlighted in blue, red and yellow the first three HSEM blocks in the Dolphins stadium rsf. For the first HSEM block the numbers 0x1b3, 1, 2, 0, 1 and 0 appear 20, 28, 32, 36, 40 and 44 bytes respectively after the word HSEM. The pattern repeats with different values in all the HSEM blocks. For the second HSEM block, we find the numbers 0x128, 1, 2, 2, 3 and 1 appearing 20, 28, 32, 36 and 40 bytes after the word HSEM and so on.

Focusing on the blue area (first HSEM block) in the screenshot, the number 0x1b3 does not seem to be an offset for UVs, as that offset is not part of any MRTS block in the rsf file. Is not the number of UV's for the first sub-mesh as it is different from 0x519 (number of faces) appearing at the first faces indices (XDNI) block. Maybe the HSEM block is not ordered sequentially.

Still not clear to me how to link the information in each HSEM block to find the corresponding offsets for the UVs.

Edited December 6 by lorak

[lorak]

The blender-api for python above version 4 needs bpy.ops.wm.obj_import(filepath = path_to_file) instead of bpy.ops.import_scene.obj(filepath = path_to_file).

Edited December 6 by lorak

[lorak]

A few screenshots of the Buccaneers stadium including the pirate ship.

Sub-meshes 2, 117 and 128 have extension objx and are not loaded into Blender. @shak-otay Maybe I need to try to make a correction, right?

Edited December 6 by lorak

[shak-otay]

30 minutes ago, lorak said:

A few screenshots of the Buccaneers stadium including the pirate ship.

Sub-meshes 2, 117 and 128 have extension objx and are not loaded into Blender. @shak-otay Maybe I need to try to make a correction, right?

Look for the first line in said objx to find the number of the H2O file they have been created from.

Then load the H2O files and check the addresses by viewing them in a hex editor.