medievil, posted Thu May 03, 2018 12:26 pm (34635)

---

PRP1986 wrote:

I've been following this (in awe as I struggle to understand scripting lol) - good work!!

trial and error, mostly error..lol
have hit a snag,, I dunno how to process the next file without the script going back to the beginning and asking for number of textures and recreating the multitex part of the header again (as well as resetting the pointers)

medievil, posted Fri May 04, 2018 8:43 am (34655)

---

hmmm...might be easier to crate a python script I can jump around in the open file (f.seek command) to get the info, place it is the output file (w.seek command (f. and w. are placeholders I can use anything I want, just easier to keep track as f and w)
this simple little example is almost all I need.. just have to recreate the subheaders.... this already offsets the data and places it, creates the main header (minus the data offset cause I haven't computed it in the code yet), it gets all the mtx filenames, computes how many there are. so it is all automated (will need one user prompt though the save file name once I get it completed although I could probnaly automate that too just using one of the filenames and cutting off the (xxxx stuff at the end
hardest thing here is converting to hex cause python is not a native hex program had to use a txt file with the multitex header in hex in it to avoid writing it one character at a time

Code:

import binascii
import os
import glob
from os import listdir
from os.path import isfile, join
files_in_dir = [ n for n in glob.glob('y:\\test1\\*.mtx*') if isfile(join('y:\\test1\\*.mtx*',n)) ]
total = len([ n for n in glob.glob('y:\\test1\\*.mtx*') if isfile(join('y:\\test1\\*.mtx*',n)) ])
print total
w=open('y:\\test1.mtx','w b')
with open('y:\\header.txt') as g:
    for line in g:
        w.write(
            binascii.unhexlify(''.join(line.split()))
        )

w.seek(0)
w.seek(20)
w.write(chr(total))

dataoffset=(total * 128)   36
print dataoffset
w.seek(dataoffset)
for file in files_in_dir:
    #with open( file, 'rb') as f:
     f=open(file,'rb')
     b = os.path.getsize(file)
     b = b - 64
     f.seek(64)
     for i in range(b):
      a = f.read(1)
      w.write(a)
 #print a,
f.close
w.close

medievil, posted Fri May 04, 2018 11:19 pm (34686)

---

I am one byte (can't seem to read it as an integer) away from having a complete python solution!! I'll likely tweak it a bit, currently have to read 3 separate txt files for some static hex. Could probably place them inside as string's ..and gotta tweak the path so you'd just have to run it from the folder you put the group of mtx files in..and gotta convert it to executable so one doesn't need python installed.
Python scripting is a lot easier than I always thought... reminds me of a cross between old standard basic and 8bit assembly...learned alot in 2 days!

medievil, posted Sat May 05, 2018 5:02 am (34690)

---

script is done!! works perfect...all one have to do is place all the like texture files in a folder (all caustic's together for example) and the script... run it and enter the folder where the textures are (mtx format that the buffy pak extractor makes) and enter the name for the final output file(all the caustic(xxxx) should be simply caustic.mtx (,mtx not needed on script)...in the meshes folder you take for example, under scoobygang folder all the willow files, name the output willow... and in a few seconds you'll have the full mtx file in a folder inside called output. Be careful not to mix in other textures probably wouldn't hurt anything BUT you never know.. you could also use this to create all new indy textures although you have to strip thee header and create a new mtx ripped format one.. not hard bu...maybe I will work on that next
Not posting yet as I am tryign to get the environment setup to turn the script into an exe

medievil, posted Sat May 05, 2018 5:08 am (34692)

---

link to example mtx I just created a few minutes ago

https://www.dropbox.com/s/s01a41rmoc6vz ... w.mtx?dl=0

PRP1986, posted Sat May 05, 2018 7:11 pm (34709)

---

Good work!

medievil, posted Sat May 05, 2018 7:45 pm (34710)

---

here it is... readme included for usage
removed link, found a bug...not in the program so much as the format that I missed (4 byte separator between textures)
if anyone grabbed it, wait, though they look correct, the produced mtx's won't decode

medievil, posted Mon May 07, 2018 3:36 am (34746)

---

still workign on it, ran across a rather perplexing obstacle... who-da thunk taking a raw 4 byte aa bb cc dd and making it dd cc bb aa would be so elusive! I need to use the data size inside the individual texure mtx's(not the total file size) and it is in reverse order (Which is fine for writing it to the main mtx, but not for calculation offset)

medievil, posted Wed May 09, 2018 3:00 am (34790)

---

have fixed my tool, still not converted it to exe yet...textures extract fine from the mtx extractor that was made to extract indy textures (that 4 byte separator I found wasn't.. it was actually texture data size so had to be specifically written the textures have more than one mipmap as well, but I'll have to study how they are packed in the indy mtx (not sure they have a header as they do on the buffy extracted ones).. ANYWay.. textures work, but they still don't work in game in the level I subbed in as well as the character model I subbed in... Indy looks like buffy, but no textures..I assumed the mesh would specify the textures to use (Same for levels, like the spanish mission..no textures, fire works though (it is using the indy caustic or partical mtx that came with indy. Pretty sure they are the same on both) here is a screen shot:

medievil, posted Wed May 09, 2018 3:32 am (34791)

---

new discovery that s also not helpful in the end... the indy exe can read the pak files (have to set pak in the default.cfg to 2).. after it reads the one it needs though it pops up an access violation map stalkwalker or something or other..., it specifically told me it couldn't read res_m01_srilanka_01.pak untill I placed it in the folder then it continued on, it read the res null pak first

medievil, posted Wed May 09, 2018 2:58 pm (34808)

---

progress... seems pc's indy res.dat is the same format and stuff as buffy's res_null.pak.. renamed it res data and placed textures in the same folder layout as buffy (not all of them yet) and....
**edit** forgert above, simply delete the res.dat, the exe will recreate it with stuff added

medievil, posted Thu May 10, 2018 5:17 am (34835)

---

not sure this is ever gonna work.. seems the "Script" for the game engine is embeded (Gscript_indy.dll on PC and packed into the xbox default.xbe) without it, I dunno the game can be made to run.. I mean levels sure, textures Eventually all of them can be worked out EXCEPT, the main playable character.. I can find no where that it lists the character. Missionlist.txt one of the cmd arguments is the player model ( like indy in a tux, indy in his university suit, etc..).. If the xbe can be unpacked (doubt) possibly an equivalent is there that can be worked with (it is , after all, x86 code)... shame too cause it really is starting to look like something

medievil, posted Fri May 11, 2018 4:24 am (34848)

---

well finally figured the character texture thing out... all the textures in the level are present...missing audio, items, game logic (can't solve the puzzle to continue on cause the item isn't there...I have to look further into the level file and see whats up)

PRP1986, posted Fri May 11, 2018 6:35 am (34849)

---

medievil wrote:

well finally figured the character texture thing out... all the textures in the level are present...missing audio, items, game logic (can't solve the puzzle to continue on cause the item isn't there...I have to look further into the level vile and see whats up)

Wow nice one how did you get the character textures to work in the end?

medievil, posted Fri May 11, 2018 7:41 am (34850)

---

You have to rename the buffy msh file (I used buffyslayer.msh ) to indianajones.msh and place buffyslayer.mtx .ban and .gin all in the main meshes/char folder, only rename the msh file though.. after that delete res.dat and they loaded when it got created back....all the textures are like that, once you place them in the layout (the map textures have to be in the same folders they would be in for buffy.. like textures/props/xxxx... you can load the map in a hex editor and see all the textures used....mine even has all the trees and bushes on it now...and yes hat is REQUIRED in the game script...I really wish I knew how to get the scripting out of the xbox xbe or could replace all the info in the indy one so that it read the buffy file names and followed he buffy script... I've still not found the control for EITHER game, the one that says play this animation and this audio file> we have the missions list placing one of the buffy missions at the top doesn't always work though

medievil, posted Sat May 12, 2018 3:58 am (34893)

---

not giving up, but progress is gonna go to a crawl...I gotta learn debugging , ida, etc... so I can figure out how to transfer/translate the scripting and my time is fairly limited so it will take a while...

medievil, posted Sat May 12, 2018 11:12 pm (34907)

---

https://www.dropbox.com/s/s7ipokral0c8jz2/mtx.zip?dl=0

working tool to create the multitex files from individual buffy pak extracted mtx files

slayerbot_, posted Mon May 14, 2018 7:23 am (34919)

---

So amazing to see someone working on this, I had to join just to say Thank You . I wish you luck with this and hopefully one day it will all be done and we will have a PC port, my dream haha

medievil, posted Tue May 15, 2018 1:04 am (34927)

---

hmmmm gonna have to investigate the rpe files.. they are definitely in xbox adpcm format (0x69) the rpe format itself is pretty easy... 01 is first byte, 4th is number of audio files... then, like the mtx, you have specific headers with name, offset location and size of file.. I'd hate to have to write a script to extract them all for conversion to regular adpcm (0x11).... one rpe has over 1000 audio files most have about 100.. thats a lot of hand converting since I don't know of a tool that can batch convert xbox adpcm to regular wav. Also been looking more into the level files, seems maybe the scripting MIGHT be there after all... their are references to dialog, characters, etc....not seeing why , for instance, on the opening level giles is not there talking to her (even if there was no audio) though...

slayerbot_, posted Tue May 15, 2018 2:13 am (34929)

---

**EDIT**
never mind, you had already done what i was suggesting

slayerbot_, posted Tue May 15, 2018 5:54 am (34932)

---

This post http://the-scholars.com/viewtopic.php?f=38&t=22730# might be helpful, the person has written a script that might do what you need in replacing the hex code. There are other programs but they are paid and dont really let you test them first. The othet thing i wanted to ask, I was looking through the hex code for the wav files and was wondering if all of the 0x69 need to be changed or just the first one. I could help you to change the values or do it for you if the script wont work i just need to know what to change haha and how to get all of the rpe files or if you can give me access to them.

medievil, posted Tue May 15, 2018 4:36 pm (34946)

---

not that easy.. to be done proper, the files need to be reencoded and new rpe's created... I can do it(I can create a python script to extract them and then to recreate the rpe.. the time consuming part is reencoding... aluigi created a reencoder on the bms site as far as I know it doesn't do multiple files at once though

slayerbot_, posted Wed May 16, 2018 2:54 am (34954)

---

It can do batch files with this code, i don't know how to make it do sub folders though.. tried but i just don't know enough.
for %%G in ("%~dp0"\*.wav) do xbadpdec.exe "%%G" "Done\%%~nxG"

When using the program for testing on the .wav files the newly created files are much larger and I'm not sure if that's an issue as well the codec is changed to PCM not ADPCM. If you use the -a option there is no change to the codec and remains the same size, its still unplayable on a system without the xbox codec installed. If you manually hex edit the file to change all the 69 to 11 MediaInfo will report it as being ADPCM and will also play on a system without the xbox codec installed on it, but I'm not sure if that's will work out as you said that wouldn't be the right way to do it.

I went back and learnt how to extract the .pak files with Luigi's QuickBMS but its just straight to .wav no .rpe so that's what I'm working with. I also noticed that the sound files extracted in the music folder are .wma matching the ones outside of the .pak files but tiny file size and no codec info when checked.

Hope I'm being of some help and not just holding you up

**EDIT**

Also have you seen this XBE Explorer http://dxbx-emu.com/downloads/ might help you find some more stuff in the XBE file

medievil, posted Wed May 16, 2018 5:47 am (34955)

---

ok, if that will do them in a batch I will write a python script(2 of them) one to extract the files and one to put them back together... size isn't an issue it will just make the rpe files larger.. the rpe's are in the dialogue folder BUT that might not get extracted, it might be part of the install.dat... I know I got them running the game in cxbx..it creates the HD structure/partitions in users/your profile/appdata/roaming/cxbx reloaded... partition 4(the folder, not the bin)is where buffy stores everything.... it will take me a few days to get the scripts working (Work and all, limited time, unfortunately) and yup, saw xbe explorer... moved on from it... ida pro, this actually might be dooable, will take a while though (months most likely at the earliest)... the whole engine main body is pretty much made of structs if it is movable, useable, a person or even some rocks, it will have a struct that gets called. Advantage for me, I have the pc and xbox version of indy to do my comparisons and plot out how best to make changes.. another advantage (I think).. the PC indy uses rtti.. which means instead of having a blue million structs, you feed it info and it creates the struct on the fly...if I figure the format out, it will be easier to translate buffy engine data cause I'd just need to make the correct data string... Thats of course, if I understood Rtti when I looked it up.

slayerbot_, posted Wed May 16, 2018 11:16 am (34960)

---

ok cool, i will keep an eye on your progress and if i see anything i might be able to help with i will try. Good luck