Messages

I've started working on a simple Stunts resource editor; stressed. Only plain text resources are supported right now, so MISC.PRE is probably the only file it can load. Compressed files are supported by re-using stunpack (as intended), but remember to change the file name (=> MISC.RES) when saving as encoding is not supported.

you may be sure beta testers won't be lacking!

The code is available in a public repository. Google Code project hosting doesn't offer XML feeds of commits, but I'll post a message here when/if I add something exciting. In order to build it you'll need a SVN client and the QT4 dev package.

Code Select Expand


svn co http://stuntstools.googlecode.com/svn/trunk/stressed/ stressed
cd stressed
qmake stressed.pro
make
src/stressed /path/to/MISC.PRE


I've only tested it under x86_64 Linux, altought that shouldn't really matter. It'll hopefully compile for Windows/Mac etc. as well using the GNU toolchain.

BTW, before I forget, dstein: are you reading the colour data while generating that shape pics for the file from somewhere?

The "colors" are in fact materials that can have simple textures. Locating and reversing this data structure would take longer than just making the list by hand. I took in-game screenshots and identified the color values...

dstien will find specially amusing the fact it is written in Fortran...

Hehe, I do find your choice of languages -- Fortran and R -- a little, um, esoteric.

And before I forget, nice find about the helmet, when we find what escape code produces the circular taillights it will be a neat addition.

Actually, there's no magic about the GTO's "round" taillights, they're just 6-sided polygons:

Those corrections mean that now we only have the mysterious "UNKNOWN" middle section to understand. Actually, I did some brute force tests while writing the parser (e.g. blanking the whole thing), and it seems that the "UNKNOWN" section has actually two parts, one with five-byte blocks and the other with three-byte blocks. The first part might have some relation with angles of visualization - for instance, wheels have "ff ff ff ff ff" sequences, and blanking some (don't remember which) values will make them disappear, while arbitrary changes might make them go on and off as the car0 structure rotates in the selection screen. As for the second part, still no clue on what it stores.

The first part does indeed appear to store values related to detail culling. I think the first list is 4 bytes per polygon, maybe max-distances? I'll have to dig deeper into this...

Now, on where we go from now... first thing I wish to do is to print better polygon maps, so it is feasible freely colorize a car. Also, I wonder about editors. One of the motivations behind the creation of the ASCII parser was to use it as an intermediate tool for feasible edition. I can think of two routes:

• Decompose the .3SH in tables of vertexes, colours, polygons and whatever else is needed, modify the intermediate as wished and re-package everything on a new .3SH with another program. That could be useful, for instance, in providing a comfortable way to tune colours without having to worry all the time about offsets - but more important would be the role of such process in the other treatment...
• ...namely, use a new parser to convert the essentials (vertex and polygons at least) data in a format that can be read by an actual 3D editor, then use another program to rebuild the .3SH. While that may sound unfeasible at first, there's a damn simple and common 3D shape format that uses tables very similar to what we already have to code polygonal shapes: the .OBJ format.

For the moment, though, I'm merely speculating. Surely dstein knows a lot more about 3D shape edition than me (for I would have found nothing without his guidance), and is a more experienced programmer as well (he doesn't have to write his parsers in Fortran  ), so he can have a more proper opinion in that respect.

I plan on making a resource editor with low-level editing and import/export of 3d mesh data...

Another special case of numIndices is the value 11 (0x0B), it takes two vertex indices to make a sphere (origin + radius). It's used to render the helmet on the Indy shape in the car selection screen:

Yup, visualization did the trick. Now we can see how the wheels are built.

It seems like we have the basic structure now:

Code Select Expand

struct SHAPE {
    BYTE numVertices;
    BYTE numPolygons;
    BYTE numPalettes;
    BYTE reserved; // Always == 0. Ignored.

    VERTEX vertices[numVertices];
    UNKNOWN unknowns[numPolygons];
    POLYGON polygons[numPolygons];

    BYTE padding[]; // -> EOF. Ignored.
};

struct VERTEX {
    WORD x;
    WORD y;
    WORD z;
};

struct UNKNOWN {
    BYTE data[8];
};

struct POLYGON {
    BYTE numIndices;
    BYTE depthIndex;
    BYTE colors[numPalettes];
    BYTE indices[numIndices == 0xC ? 6 : numIndices];
};

The value depthIndex in the POLYGON struct is used to override the game's depth clipping. The barn shape, for example, has details drawn on a plane at the exact same depth. Without setting custom depth-indices the details will flicker:


I don't think shapes stores animation data (windmill rotations, "Joe's" flashing sign). They may be hardcoded...

Moreover, the word "numIndices" is 0x0c, but there are only six indices to edit... and if you try to deform a tyre by extending it in one direction the wheel will be suitably adjusted and a second distortion, symmetrical in relation to the origin vertex, will take place as well. Thus, there is *something* else involved in wheel drawing.

Interesting. It sounds like numIndices can also be a code for using other polygon primitives, or an escape code for referencing previous polygons...

BTW, FYI, counting printable-byte colour strings reveals we have (a few more than) 223 polygons in STCOUN.3SH. Of those 223, 148 are in car0 and 66 in car1...

Remember that each resource is an independent entity. All shapes have their own SHAPE structure with header values, it seems to add up.

I attached a very sloppy (and sleepy!) spreadsheet chart which may serve as a rough guide for the time being...

Wow! Again, well done. It'll sure prove useful.

[CTG]

Nice find, Duplode! According to my current understanding of the mesh structure, the number of color variations is a part of the header:

Code Select Expand


struct SHAPE {
    BYTE numVertices;
    BYTE numPolygons;
    BYTE numPalettes;

    [...] // Vertices and other unknown data?

    POLYGON polygons[numPolygons]
};

struct POLYGON {
    WORD numIndices;
    BYTE colors[numPalettes];
    BYTE indices[numIndices];
};


This means that the number of palettes isn't fixed. We can use the simple exp_ shapes to verify these assumptions:



There are four explosion shapes, they all have one polygon, so the number of vertices per polygon appears to be variable. For STCOUN.3SH, exp0 and exp3 are quads with the default color and a darker shade. exp1 is a triangle with the default color and exp2 is a single black line.

While this makes fine-tuning possibilities endless (bi-colour cars!!), it also might make colour changes a headache if one lacks a good hex editor (I'm comfortable with KHexEdt, under Linux obviously) - even more because the number of panels and resource lengths vary from car to car.

Yes, the field of free hex editors is a sad state of affairs. It looks like Okteta will replace the unmaintained KHexEdit in KDE4. I tend to use shareware Windows hex editors under Wine to get advanced features such as scripting and data structure templates.

CTG - My comment was aimed at cheaters, I'm sorry if I offended anyone here.

Edit: Added polygon struct pseudo code.

I don't see how reversing visual game assets - bitmaps and 3d shapes - is a threat to racing competitions. The information about car tunings, replays, high scores and tracks have been available for quite a while. From my experience with other game communities I've noticed that there's a distinct correlation between the urge to claim deceitful achievements beyond one's capabilities and sub-average intelligence.

My motivation for doing this is that I want to make new cars from scratch. Reversing is fun as well.

Dstien, my dear namesake  , or Zak, please enlighten me: I succeeded to unpack GAME2.P3S under Linux and to open it under Vim in hex-mode... but how one can tell what parts of the file are a "TRUK" shape, or a "BARN" or an "ELRD"?

As discussed earlier, all game files have a common structure for storing multiple resources. For each id name there's a 4-byte offset into the data section of the file. At offset 0x04 is the number of resources, next is the list of id names at offset 0x06. Each id is four bytes, so the following list of offsets is located at 0x06 + numRes * 4. The corresponding data offsets are 4-byte integers, ending at 0x06 + numRes * 4 + numRes * 4, which is the base offset for the data section.

To locate a shape first find its index in the list of ids. "truk" is the 40th resource in GAME2.3SH. The shape's data offset is thus the 40th entry in the offset list. Add this value to the base data offset and you'll have the position of the 3d shape.

0000000: 6c85 0000 3b00 6261 726e 626f 6174 6272  l...;.barnboatbr  <- Number of resources
0000010: 6964 6361 6374 636c 6431 636c 6432 636c  idcactcld1cld2cl
0000020: 6433 656c 7264 656c 7370 6578 7030 6578  d3elrdelspexp0ex
0000030: 7031 6578 7032 6578 7033 666c 6167 6761  p1exp2exp3flagga
0000040: 7373 676f 7569 676f 756f 676f 7570 6869  ssgouigouogouphi
0000050: 6731 6869 6732 6869 6733 6869 6768 6c61  g1hig2hig3highla
0000060: 6b63 6c61 6b65 6c63 6f30 6c63 6f31 7061  kclakelco0lco1pa
0000070: 6c6d 7261 6d70 7263 6f30 7263 6f31 7264  lmramprco0rco1rd
0000080: 7570 7265 7374 7365 6c72 7365 7374 7369  uprestselrsestsi
0000090: 676c 7369 6772 7372 616d 7465 6e6e 7472  glsigrsramtenntr
00000a0: 6565 7472 756b 7769 6e64 7a62 616e 7a62  eetrukwindzbanzb  <- "truk" at index 39
00000b0: 6f61 7a62 7269 7a62 726e 7a65 6c72 7a65  oazbrizbrnzelrze
00000c0: 7370 7a67 6173 7a6c 636f 7a70 616c 7a72  spzgaszlcozpalzr
00000d0: 616d 7a72 636f 7a72 6475 7a72 6573 7a73  amzrcozrduzreszs
00000e0: 6572 7a73 6573 7a73 7261 7a74 656e 7a77  erzseszsraztenzw
00000f0: 696e 0000 0000 ec01 0000 9e06 0000 a208  in..............  <- End of id name list, start of offset list
0000100: 0000 480d 0000 840e 0000 4a0f 0000 ee10  ..H.......J.....
0000110: 0000 f012 0000 3e14 0000 7e14 0000 be14  ......>...~.....
0000120: 0000 0215 0000 5a15 0000 0016 0000 4a1b  ......Z.......J.
0000130: 0000 b01b 0000 f41b 0000 3a1c 0000 6a1c  ..........:...j.
0000140: 0000 9a1c 0000 ca1c 0000 fa1c 0000 261d  ..............&.
0000150: 0000 541d 0000 5829 0000 702a 0000 e02c  ..T...X)..p*...,
0000160: 0000 982e 0000 9c3a 0000 b43b 0000 423e  .......:...;..B>
0000170: 0000 1844 0000 ba45 0000 884d 0000 ee4e  ...D...E...M...N
0000180: 0000 5450 0000 0052 0000 1e53 0000 1e54  ..TP...R...S...T  <- truk's data offset at index 39
0000190: 0000 4e59 0000 9a62 0000 ba63 0000 aa66  ..NY...b...c...f
00001a0: 0000 c467 0000 e468 0000 fc69 0000 606a  ...g...h...i..`j
00001b0: 0000 826d 0000 7871 0000 fa72 0000 c873  ...m..xq...r...s
00001c0: 0000 be77 0000 3479 0000 9a7b 0000 e27c  ...w..4y...{...|
00001d0: 0000 a07f 0000 6280 0000 ce80 0000        ......b.......   <- End of offset list, base data offset
[...]                                                                  truk's content is at 0x01DE + 0x541E = 0x55FC

Now we know where truk's data starts, but not where it ends as we don't know how to parse the 3d shape data yet. To find the end just use the start offset of the next resource.

I doubt that DSI licensed a format for storing straight-forward bitmaps. The filename extensions are likely just coincidences, the 8+3 naming scheme have led to many collisions. These formats may however have been re-used in other games made by DSI/Unlimited Software at the time, I'm eager to know if this is the case, especially the compression.

The compression can be variable-length encoding or run-length encoding. A compressed file may contain another compressed sub-file, hence allowing both compression schemes to be combined recursively. The total size of all game files uncompressed is 2.5 MB (BB 1.1), this would need four DD 3.5" floppy disks, as opposed to two as the game was shipped on (1.2 MB).

I have registered a Google Code hosted project for my tools, the first one being stunpack, a simple command line program for unpacking packed resources (PRE/P3S/PVS/PES) and code files (COD/CMN/DIF). BB 1.0 files are not supported. They differ from 1.1 files, but I haven't investigated it further.

Here's a pre-release Win32 EXE file. I've only tested it under Wine, so please check if it works on an actual Microsoft system as well. You can do this by unpacking resource files and move the original packed ones out of reach. The game will thus load unpacked ones instead. If the game works as normal, the decompression was successful. Please note that some bitmaps will appear distorted. This is not caused by the decompression, but because some bitmaps are stored in vertical lines instead of the regular horizontal lines. This is probably done to get optimal byte-runs.

• stunpack-0.1.0-rc-win32bin.zip
  
  

Naming for fully decompressed resource files:

Code Select Expand


Misc variables  PRE -> RES
3d shapes       P3S -> 3SH
Bitmap images   PVS -> VSH
Icons(?)        PES -> ESH


Example:

Code Select Expand


stunpack stuntsdir/GAME1.P3S stuntsdir/GAME1.3SH
ren stuntsdir/GAME1.P3S stuntsdir/_GAME.P3S


Now we can start dissecting the actual data stored in the resource files. I tried making a new car by using the existing shape "TRUK" found in GAME2.P3S.
It works, but the mesh will be corrupted as there's no dynamic wheels to transform.

Thanks! I have reversed the first decompression pass, enough to unpack PRE and code files (COD/CMN/DIF), the other files use an additional layer of compression which I'm currently looking into. P3S and PVS files can contain multiple shapes and bitmaps, organized in the same structure as RES files. The game accepts uncompressed data files (P3S => 3SH, PVS => XVS), so it doesn't look like there's need for an encoder.

My interpretation of the RES file structure:

Code Select Expand


WORD lenRemainder
BYTE lenMultiplier   // total = (lenMultiplier * 0xFFFF) + lenRemainder
BYTE unknown         // Passes == 0?
WORD numVars

CHAR ids[numVars][4]
DWORD offsets[numVars]

BYTE values[]       // Rest of file


A RES file has numVars variables, each with a 4-chars identification and an offset pointing to the data in values[]. Text is null-terminated strings, the structure of other value types depend on the id.

Here's an example of the unlock questions and answers from MISC.PRE (more on the decompression later):

Code Select Expand


Num Id   Offset   Value
--- ---- -------- -------
  1 ea00 000002C2 camera
  2 ea01 00000314 destructive
  3 ea02 00000368 conventions
  4 ea03 000003BC previewing
  5 ea04 000003FC proper
  6 ea05 00000426 cables
  7 ea06 00000460 loop
  8 ea07 000004AC displaying
  9 ea08 000004EB you
10 ea09 00000537 query
11 ea0a 00000571 button
12 ea0b 000005AC directory
13 ea0c 000005F9 around
14 ea0d 00000648 currently
15 ea0e 00000687 selected
16 ea0f 000006D4 immediately
17 ea0g 00000727 track
18 ea0h 00000766 three
19 ea0i 000007A0 110
20 ea0j 000007D7 buttons
(...)
51 eq00 00000292 106 1 @       mode while in this]viewing mode.]
52 eq01 000002C9 1 122 ...and @             stunts that]can smash your car to pieces in...]
53 eq02 00000320 2 5 8 ...control your car, as well as]the @            of using the...]
54 eq03 00000374 3 134 ...editor'). In addition,]@           might give you a]better...]
55 eq04 000003C7 4 8 1 @       direction. If you get on]the track...]
56 eq05 00000403 5 123 hit the @      and crash...]
57 eq06 0000042D 6 3 7 while you are driving through]the @     ...]
58 eq07 00000465 7 2 1 @            the current car, an]acceleration curve (showing...]
59 eq08 000004B7 8 5 7 Just your everyday type of]vehicle. @    ...]
60 eq09 000004EF 9 3 9 ...course, you'll be interrupted]while driving with a @      ...]
61 eq0a 0000053D 103 5 your driving performance. This]@       is...]
62 eq0b 00000578 112 1 @          . Highlight the]desired replay...]
63 eq0c 000005B6 122 1 @        ,you can choose to race]any one at a time. Each...]
64 eq0d 00000600 134 11this screen, you'll see a bird's]eye view of the @           ...]
65 eq0e 00000652 146 1 @          piece is the area upon]which it...]
66 eq0f 00000690 2 8 1 @            on the standard track]by clicking on the car...]
67 eq0g 000006E0 3 2 1 @      selection screen. Your]goal is to be on it. Your real...]
68 eq0h 0000072D 4 4 4 ...time than missing @      ]pieces of regular...]
69 eq0i 0000076C 6 5 8 ...out at the peak, while]more than @    MPH]
70 eq0j 000007A4 7 5 7 ...retail price), and a set]of @        ...]
(...)