Yesterday at 06:23 AM1 day ๐งต [REQUEST] Noesis plugin for XMX/LDMX โ mesh corruption (index/stride issue)Hi,I'm reverse engineering a proprietary XMX / LDMX model format and I'm getting consistent mesh corruption when parsing it.I'd like help identifying the correct vertex layout and eventually building a Noesis plugin (.fmt or Python) to export to FBX/OBJ.๐ฆ Format info (confirmed)Header:4C 44 4D 58 โ "LDMX"Chunks found:EMAN (likely "NAME", reversed)MNXT (textures)VPRG (vertex block)INDX (index buffer)๐น Vertex dataFloat data detected:E0 FE 48 BE โ -0.196 48 E7 E7 BE โ -0.453 80 09 A7 3C โ 0.020Likely:Position (float3)followed by normals / UVs / etc.โ Unknown:vertex stride (suspected 32 / 36 / 40 bytes)๐น Index buffer58 44 4E 49 โ "INDX"Appears to be uint16 (little endian)Example:01 00 02 00 03 00โ ๏ธ ProblemMesh is corrupted:degenerate triangles like:22 00 22 00 22 00 25 00 25 00 25 00Result:collapsed facesbroken topology๐ง Suspected causesWrong vertex strideIncorrect index offsetVertex count mismatchPossible mixed endianness๐ TexturesPaths inside file:d:\FS\tex_zb01.zip/:xxxx.ddsโ DDS inside ZIP archives๐ฏ What I needHelp identifying correct vertex layout / strideEventually a Noesis plugin that:reads vertices correctlyfixes meshexports to FBX/OBJ๐งช I can provideSample filesHex dumpsTest resultsโ QuestionHas anyone seen a similar format or can help determine the correct vertex layout / stride?Best result, but it's not entirely correct; it always reads shitty padding or interspersed garbage. One or more triangles always come out wrong, but it's the best versionโthe one where the mesh turns out almost perfect...Almost all the files to be converted are located at: https://github.com/randalfcastro-tech/xmx-testing from inc_noesis import * import noesis import rapi import struct # ========================= # REGISTER # ========================= def registerNoesisTypes(): handle = noesis.register("XMX REAL FINAL STABLE", ".xmx") noesis.setHandlerTypeCheck(handle, xmxCheckType) noesis.setHandlerLoadModel(handle, xmxLoadModel) return 1 def xmxCheckType(data): return 1 if data[:4] == b"LDMX" else 0 # ========================= # FIND REAL REGION # ========================= def findRealMeshRegion(data): vprg = data.find(b"VPRG") if vprg == -1: return -1, -1 xtrv = data.find(b"XTRV", vprg) xdni = data.find(b"XDNI", xtrv) return xtrv, xdni # ========================= # VERTICES (REAL) # ========================= def parseVertices(data): xtrv, xdni = findRealMeshRegion(data) if xtrv == -1 or xdni == -1: print(" Missing real mesh blocks") return None start = xtrv + 4 end = xdni bs = NoeBitStream(data) bs.seek(start) verts = [] norms = [] uvs = [] stride = 32 MAX_VERTS = 200000 while bs.tell() + stride <= end and len(verts) < MAX_VERTS: try: x = bs.readFloat() y = bs.readFloat() z = bs.readFloat() nx = bs.readFloat() ny = bs.readFloat() nz = bs.readFloat() u = bs.readFloat() v = bs.readFloat() # sanity check (evita basura) if abs(x) > 10000 or abs(y) > 10000 or abs(z) > 10000: break verts.append((x,y,z)) norms.append((nx,ny,nz)) uvs.append((u,1.0-v)) except: break print(" Vertices:", len(verts)) return verts, norms, uvs # ========================= # INDICES (REAL) # ========================= def parseIndices(data): _, xdni = findRealMeshRegion(data) if xdni == -1: print(" No XDNI") return None bs = NoeBitStream(data) bs.seek(xdni + 4) indices = [] MAX_INDICES = 500000 while bs.tell() + 2 <= len(data) and len(indices) < MAX_INDICES: indices.append(bs.readUShort()) print(" Indices:", len(indices)) return indices # ========================= # STRIP โ TRIANGLES # ========================= RESTART = 0xFFFF def stripToTriangles(indices, vertCount): tris = [] flip = False for i in range(len(indices)-2): a = indices[i] b = indices[i+1] c = indices[i+2] if a == RESTART or b == RESTART or c == RESTART: flip = False continue if a == b or b == c or a == c: flip = not flip continue if a >= vertCount or b >= vertCount or c >= vertCount: continue if flip: tris.extend([b, a, c]) else: tris.extend([a, b, c]) flip = not flip return tris # ========================= # BUILD # ========================= def buildMesh(verts, norms, uvs, indices): tris = stripToTriangles(indices, len(verts)) # sanitizar clean = [] maxVert = len(verts) for i in tris: if not isinstance(i, int): continue if i < 0 or i >= maxVert: continue clean.append(i) clean = clean[:(len(clean)//3)*3] if len(clean) < 3: print(" No valid triangles") return vb = b''.join([struct.pack("3f", *v) for v in verts]) nb = b''.join([struct.pack("3f", *n) for n in norms]) ub = b''.join([struct.pack("2f", *u) for u in uvs]) rapi.rpgBindPositionBuffer(vb, noesis.RPGEODATA_FLOAT, 12) rapi.rpgBindNormalBuffer(nb, noesis.RPGEODATA_FLOAT, 12) rapi.rpgBindUV1Buffer(ub, noesis.RPGEODATA_FLOAT, 8) try: ib = struct.pack("%dH" % len(clean), *clean) except Exception as e: print(" PACK ERROR:", e) return rapi.rpgCommitTriangles( ib, noesis.RPGEODATA_USHORT, len(clean), noesis.RPGEO_TRIANGLE, 1 ) # ========================= # MAIN # ========================= def xmxLoadModel(data, mdlList): ctx = rapi.rpgCreateContext() parsed = parseVertices(data) if not parsed: return 0 verts, norms, uvs = parsed indices = parseIndices(data) if not indices: return 0 print(" Building mesh") buildMesh(verts, norms, uvs, indices) mdl = rapi.rpgConstructModel() mdlList.append(mdl) return 1 fmt_parser_xmx.py Edited yesterday at 07:02 AM1 day by Ralp1670
Yesterday at 07:02 AM1 day Supporter degenerated triangles: just a wild guess (without checking the details): maybe the indices of those models with that problem are DWords, not shorts?
Yesterday at 07:18 AM1 day Author 15 minutes ago, shak-otay said:degenerated triangles: just a wild guess (without checking the details): maybe the indices of those models with that problem are DWords, not shorts?๐ NOT DWORD (uint32)โ These models correctly use UINT16.
Yesterday at 08:35 AM1 day Supporter 1 hour ago, Ralp1670 said:No, son.Your disrespect forces me to leave the thread. Good luck.
Yesterday at 08:44 AM1 day Author 6 minutes ago, shak-otay said:Your disrespect forces me to leave the thread. Good luck.6 minutes ago, shak-otay said:Your disrespect forces me to leave the thread. Good luck.Friend, I thought you would ask questions or present arguments, but you are taking things for granted that simply aren't true; even if it sounds like a clever answer, it isn't backed by factsโand Iโve tried to be as polite as possible...
Yesterday at 10:55 AM1 day Localization 4 hours ago, Ralp1670 said:I'm reverse engineering a proprietary XMX / LDMX model format and I'm getting consistent mesh corruption when parsing it.Can you tell us which game (?) uses this .xmx format?
Yesterday at 02:19 PM1 day Supporter Since there is FFFF strip terminator it could be some xbox / 360 game. But now it's a history. Also i can't see any samples.
22 hours ago22 hr Hi everyone,First of all, Iโd like to apologize for my previous tone โ that wasnโt appropriate, and I appreciate the time youโre taking to help.Regarding the suggestion about DWORD indices:Iโve double-checked the data and Iโm fairly confident the index buffer is uint16, based on:Buffer size alignment (2 bytes per index)Values staying within vertex rangeNo valid mesh produced when interpreting as uint32However, the issue with degenerate triangles still persists, so I believe the problem is more likely related to:Incorrect vertex stride (possible padding or interleaved data)Misaligned vertex buffer startOr mixed vertex layouts within the same blockIโm currently getting almost correct meshes, but with occasional corrupted triangles, which suggests Iโm very close but still reading some garbage data.To help debugging, Iโve provided sample files here:https://github.com/randalfcastro-tech/xmx-testingIf anyone has seen a similar format (especially with VPRG/XTRV/XDNI chunks), Iโd really appreciate any insight.Also answering the question:Iโm still trying to identify the exact game/engine using this format โ Iโll update as soon as I confirm it.Thanks again for your time.
21 hours ago21 hr the vertex data length in this format is not fixed; according to my inspection, it can be 28/32/40 Edited 21 hours ago21 hr by kurokozerefx
21 hours ago21 hr From what I can see, you need to parse the "VPRG" block, which seems to be the info for each submesh. The count value is in the file header at offset 0x38, and each VPRG entry looks to be 0x48 bytes, although I haven't checked every file to confirm that exactly. Each VPRG entry contains offsets, counts and stride values for each submesh's vertex and face data, and probably other info as well. The stride varies between submeshes, and the face indices start from 0 again for each submesh, so you have to read each vertex/face block separately for each submesh.Also, how do you not know which game/platform it is? Where did you get the samples? I have seen similar formats in other games, but can't recall off the top of my head which ones.
21 hours ago21 hr Hi, thanks a lot for the detailed explanation โ that clarifies things a lot.I didnโt realize the VPRG block was driving the submesh layout. Parsing each submesh separately with its own stride and index buffer starting at 0 makes total sense now, especially with the 28/32/40 variation.Iโll try to restructure my importer around VPRG entries (0x48 each) and see if that fixes the remaining corrupted triangles โ thatโs probably where my misalignment is coming from.Regarding the game: I got these samples from extracted assets, but Iโm still in the process of confirming the exact title/engine. I should be able to share more details once I verify it properly.By the way, have you already implemented a Noesis script for this format, or something similar? Even a partial example (especially how you're handling the VPRG parsing and variable strides) would be extremely helpful for getting this fully correct.Also, have you tested this approach on larger/more complex files? I'm wondering if there are still edge cases when multiple submeshes with different layouts are involved.Thanks again โ really appreciate the insight!
20 hours ago20 hr No, I haven't got any Noesis script for these. I was just basing my information on a quick look at some of the files, and I could see stride values that matched the vertex data, and face counts for each submesh. A few bits of info for each VPRG entry - note that the offset values all seem to need 0x10 adding to them to get the correct offset.Offset 8 - Vertex countOffset 0x10 - Offset to start of submesh vertex dataOffset 0x2c - Vertex strideOffset 0x40 - Offset to start of submesh face indicesOffset 0x44 - Face index count
4 hours ago4 hr 8 hours ago, kurokozerefx said:fmt_xmx.pyThat's about it.14 hours ago, DKDave said:No, I haven't got any Noesis script for these. I was just basing my information on a quick look at some of the files, and I could see stride values that matched the vertex data, and face counts for each submesh. A few bits of info for each VPRG entry - note that the offset values all seem to need 0x10 adding to them to get the correct offset.Offset 8 - Vertex countOffset 0x10 - Offset to start of submesh vertex dataOffset 0x2c - Vertex strideOffset 0x40 - Offset to start of submesh face indicesOffset 0x44 - Face index countHi, thanks again for all the detailed information โ especially the clarification about the VPRG block and per-submesh stride, that helped a lot.Iโve been testing the script that was shared, and while it works on some files, Iโm still running into a very common issue on others. The typical error I get is:RuntimeError: Normal buffer would have been read out of bounds by provided indices.From what I can see, this usually happens when:Indices reference vertices outside the current vertex bufferTriangle strips are interpreted incorrectly (TRIANGLE_STRIP vs TRIANGLE)Or there is still some misalignment between vertex and index buffers (likely due to stride/offset per submesh)Iโve collected several problematic samples here:xmx-testing/bugs at main ยท randalfcastro-tech/xmx-testing ยท GitHubThese are the files that consistently trigger that error, even when the mesh looks almost correct otherwise.Based on your explanation, I suspect this is happening in cases where:Multiple submeshes (VPRG entries) use different strides (28/32/40)The parser isnโt fully isolating each submesh (vertex + index buffers starting from 0)Or some vertex layouts include padding/interleaved data that shifts offsets slightlyIโm currently restructuring my importer so each VPRG entry is treated as a completely independent submesh (with its own vertex buffer, stride and index base), but Iโm still seeing edge cases where some meshes break.So I wanted to ask:In your approach, do you strictly reset and bind buffers per submesh before committing triangles?Have you needed to sanitize/filter indices before calling rpgCommitTriangles, or should they always be valid if parsed correctly?And most importantly โ do you happen to have a more complete Noesis script for this format (even if not fully polished)? Seeing a full implementation of VPRG parsing + variable strides would really help resolve these remaining edge cases.Once I confirm everything is working correctly, Iโll also be happy to share the exact game/engine these files come from โ I just want to make sure the format is fully understood first.Thanks again, really appreciate the help ๐ Edited 4 hours ago4 hr by Randalf2theReturn
2 hours ago2 hr Supporter I can confirm it's xbox game. It's in header. Why you don't tell what game it is? Are you selling it or what? Because it seems so. Or maybe i am paranoid๐คฃ
23 minutes ago23 min From looking at the files a bit more - the "RTAM" section at the start (for materials), seems like it needs to be parsed first as it has pointers to the submesh info for that material. The count at offset 0x38 in the header is the number of "RTAM" sections (0xe8 bytes per entry). There are more submesh entries than material entries, so some of them reference more than 1 submesh for that material.This is just a test using hardcoded offsets for the first of your files that don't work - this should be all 86 submeshes for st2_h_boss_die.xmx:
Create an account or sign in to comment