.SHP models¶

Every ship, station, weapon, asteroid and piece of debris in the game is a .SHP file in resource.hog.

sltool shp info <model>                 # header flags, bounds, arcs, parts, levels, turrets
sltool shp chunks <model>               # the raw chunk stream
sltool shp check <model>                # validate indices, parents and bounds, and write it again
sltool shp obj <model> <out.obj> [--lod n]
sltool shp gltf <model> <out.gltf> [--lod n] [--textures <tcachehw.dat> <palette.tga>]
make models                             # export every model to game/models
make check-models                       # validate every model, and write each again

Chunk stream¶

A model is a flat sequence of chunks with no nesting. Each is a 6-byte header followed by its records, and the stream ends with a chunk whose tag is 0xFFFF.

All fields are little-endian, unlike the .HOG container around them.

record_size is the format's versioning mechanism. Older exporters wrote shorter records, and the loader copies min(record_size, sizeof(struct)) bytes per record, leaving the rest of the destination untouched. A reader must do the same; this one zero-fills, so a field a later exporter added reads as zero in a file written by an earlier one.

The loader locates a chunk by scanning forward from a cursor until the tag matches or the terminator is reached. A miss leaves the cursor where it was, so a chunk the exporter omitted is simply skipped and the next request still finds what follows. Chunks must therefore appear in the order the loader asks for them, because a miss never rewinds.

The loader reads only the terminator's tag: model_take_chunk (0x004A2EB0) stops at 0xFFFF (0x004A2ECC) without reading the terminator's record size or count. A file loads the same whatever the two hold, even where they count more bytes than follow, and where the file ends after the tag. The reader stops at the tag too.

Tags¶

Order¶

header, parts, then for each part:
    levels, nodes, attachments, clips, point lists, trigger polygons
    for each level:       vertices, faces, materials
    for each node:        face list
    for each clip:        keyframes, events
    for each point list:  points
firing arcs

Every model follows this order and ends with the terminator at the last byte of the file.

Writing¶

shp.Model.write writes a model as the loader reads it: its chunks in the order above, each record at the size its tag has in the model's file, and the terminator with a record size and count of 0. In every shipped model each tag has one record size, and a tag's chunk stands wherever the loader asks for one or nowhere, with a count of 0 where there are no records. The older exporters wrote no point lists, trigger polygons or firing arcs at all. Model.parse keeps each tag's size, or that the file has none of it (shp.RecordSizes), and the writer cuts each record short to that size or fills it out with zeros. A model built from scratch gets whole records, each its type's size (shp.Record).

The attachment records of 136 and 168 bytes hold 12 and 44 bytes past the 124 the engine keeps, which the loader never copies (model_take_chunk, 0x004A2EB0). Unknown: what they are; they are zero in every shipped model. The reader keeps them with each attachment (shp.PartData.attachment_tails), and the writer writes them back after it, at a record size with room for them. Trigger polygons are written back as the file holds them. So every shipped model, written again, comes back byte for byte: sltool shp check checks it model by model, and make check-models for the whole installation.

model_load (0x004A44D0) asks for no tag outside the table above, so its search passes over a chunk of any other tag wherever it stands. No shipped model has one. The reader keeps such chunks whole (shp.Model.unnamed_chunks), each with how many chunks of named tags come before it, and the writer writes each after as many of its own, or before the terminator where it writes fewer.

A file laid out as no shipped model is comes back as the loader reads it, which it reads the same, rather than byte for byte. It loses a chunk of a named tag the loader never asks for, header records after the first, bytes after the terminator, and bytes of a record past its type's size, other than an attachment's. Chunks of one tag with two record sizes come back at the larger, a terminator whose record size or count is not 0 comes back with both 0, and a chunk the file leaves out where the loader asks for one comes back empty. Unverified: that nothing after model_load reads a header record past the first.

The writer fails rather than write a file the loader would misread: records of a tag the model's sizes leave out, a chunk of more records than its header can count (65535), or a part whose nodes do not each have their face list. It also fails for a chunk kept whole whose tag the table names, or is the terminator's, or whose bytes are not its records'. At a record size with room for tails, it fails for a part whose attachments have tails, but not one each.

A model edited after parsing keeps its file's record sizes, so a field past its tag's size is lost without a word. In a shipped model whose attachments are 100 bytes, a gun muzzle added loses its gun type (0x64), and a light its range and brightness (0x74, 0x78). record_sizes = .{} gives the model whole records and a chunk for every tag, as one built from scratch has, without its attachments' tails.

Records¶

Offsets below are within a record. Only the fields this project reads are listed; the rest are noted in § Unread fields.

Header (tag 0x00)¶

Part (tag 0x01)¶

A part is a hull section, cockpit, turret, engine, door or similar. Parts form a tree and each carries its own levels of detail.

Part flags at 0xF0:

Attachment point (tag 0x09)¶

A point on a part where the engine mounts something. Exporters wrote records of 100 to 168 bytes; the engine keeps 124 bytes of each. Unknown: what longer records hold past them, which are zero in every shipped model and which the reader keeps (Writing).

The engine's attachment table, filled when the game starts, gives the models and sprites for each kind and id; src/engine/game/create/models.zig transcribes it (make model-tables). Kind 0 holds missiles and their pods, 1 guns and turrets, 4 flare and light sprites, 5 cargo and fuel pods. Kind 3 is a gun's muzzle: an object takes one gun for each, of the type at 0x64, and its muzzle flash is drawn there. Kind 7 is where a spinning gun's spent cases fly from (Guns). Kind 6 is a cockpit's eject point: the flash of a pilot's ejection goes off at the last, and the pod shoots out along its Z axis (Ejection). Kind 8 is a launch point, where a ship stands to launch from the model, turned as the point is: a carrier's torpedo tubes and the Reliant's hangar hold them (Launches). Kind 9 is a docking point, turned as a ship docks there: a ship's own, its first, and a station's ports, counted part by part (Docking). Kind 2 is an engine's glow (Rendering).

For kinds 1 and 5 the engine mounts the model as an object of its own, hanging from the part's node, whose components join the owner's.

Animation clip (tag 0x0A)¶

One of a part's animation tracks (Animation). Its keyframes and events follow the level geometry, one keyframe chunk and one event chunk for each clip in turn. Older exporters wrote 8-byte records, which stop two bytes into the name.

Keyframe (tag 0x0B)¶

Clip event (tag 0x0C)¶

Point list (tags 0x0D, 0x0E)¶

A part can carry lists of points on its mesh, which the game's asserts call point lists. Each 0x0D record is a list's kind, a u32, and each list's points follow in a 0x0E chunk after the part's clips. model_load keeps a list as {kind, count, points}, and node_point_group (0x004ADD50) finds a part's list of a kind. A point:

The kinds the game reads:

The Ripper's orders take a part's first list, whatever its kind, which in the shipped models is kind 6: on each of the Ripper's back pincers, one point, where its beam comes from; and on a cargo pod's part Cargo pod, four, the middles of whose two pairs its beams reach for (The Ripper).

shp.PointList holds a list, and sltool shp info counts each part's lists.

Firing arc (tag 0x10)¶

One for each of the model's components, in the order the object lists them: the directions a turret standing on the component may fire in. Some exporters wrote 12-byte records, with no mask, which the loader leaves zeroed.

Level of detail (tag 0x02)¶

One 4-byte record per level, up to nine per part, holding only the distance beyond which the level applies: 0 for single-level parts, otherwise a rising sequence such as 5000, 10000, 15000. The geometry follows in the vertex, face and material chunks, in level order.

Vertex (tag 0x04)¶

Face (tag 0x03)¶

Every record is one triangle.

Shading modes: 0 untextured, 1 wire, 2 untextured and added, 3 unlit, 4 unlit and added, 5 unlit and blended by alpha, 6 lit, 7 lit with a highlight, 8 lit and added. Mode 1 draws lines rather than a filled triangle. The sub-mode picks mode 7's highlight and means nothing to the rest. Rendering gives what each draws.

The loader merges a fan's records into one polygon when each later record's normal lies within a dot product of 0.999, about 2.6 degrees, of the first's; the Direct3D driver draws the records of a strip, or of a fan left unmerged, together (Rendering). Each record is already a complete triangle of its polygon, so treating every record as its own triangle renders the same surface. 72-byte records stop before the polygon fields and are always plain triangles.

A face's front is the side (v1 - v0) x (v2 - v0) points to. A 3 record lists its last two corners the other way round: its front is the side of (v2 - v0) x (v1 - v0).

Tree node (tag 0x07)¶

A part's collision tree, the root first. The engine descends it to find which part of a ship another object has hit (0x0049BD30), in place of the sphere test the two objects' own radii give.

Each node is followed later in the file by a node face list (tag 0x08, one u32 a record) holding the faces inside its box, as indices into the part's first level. A node with faces is a leaf and its children are not read; the loader decides by the list, not by the -1s, and two models have a leaf whose child fields hold something else.

One model's records are 64 bytes and stop before the children. Its single node is a leaf, so nothing reads them.

148 of the 421 models carry a tree, 6582 nodes in all: capital ships, stations and gates, which are also the models whose objects list components. Fighters carry none and collide as spheres.

Material (tag 0x06)¶

A single NUL-terminated 64-byte texture name without its extension. The engine looks it up in the texture cache with a context-dependent prefix: g while the loadout screen preloads ships, r for its missile and gun loops, bare in flight, and a second l<name> lookup when the part's 0x80 flag is set on multitexture hardware.

Bounding boxes¶

The box stored in each part is derived from its finest level's vertices, but not always in the part's own frame. In the shipped models it is one of:

• the level-0 vertex extent as stored, in most parts;
• that extent after applying the part's orientation matrix;
• the same box up to an axis swap or reflection the record does not describe;
• zero-sized, never filled in;
• a different box.

sltool shp check classifies each part rather than requiring a match, since all five occur in shipped, working models. The vertices are authoritative; the stored box is a hint.

Coordinate frame¶

The model frame is X lateral, Y down, Z forward. Neither axis direction is recorded in the file; both are settled by what the parts are named and where they sit:

• every part named Lower, bottom or under is at positive Y;
• most parts named cockpit or canopy are at negative Y and positive Z;
• most parts named engine, exhaust, thruster, rear, back or aft are at negative Z;
• most parts named nose or front are at positive Z.

So +Y points at the ship's belly and +Z out of its nose. A model loaded without accounting for this is upside down.

Righting it is a half turn about the forward axis: negate X and Y, keep Z. That keeps the nose on +Z, where a viewer's default camera looks, and does not mirror the model, since negating two axes keeps the determinant positive. Negating Y alone would mirror it; negating Y and Z would face it away from the camera.

sltool shp obj applies that half turn to positions and normals; --model-space writes the coordinates exactly as the file stores them. shp info and shp check always report model space. The export leaves out wire faces and caps, and swaps the last two corners of odd strip members so that every face winds alike.

sltool shp gltf writes glTF 2.0 the same way (shp/to_gltf.zig), with each part a node under the node of the part it hangs from, moved by its origin less its parent's and not turned: a part's orientation sets the axes its animation turns it about, and leaves its mesh as it is at rest (objects.Model.placeFor). Each attachment is an empty node under its part's, named as sltool shp from-gltf reads it, and each material names its picture, <material>.png, which --textures writes from the texture cache. glTF numbers texture coordinates from the top, as this format does.

Wavefront OBJ also numbers texture coordinates from the bottom up, the opposite of this format, so the exporter emits 1 - v.

Positions are in model units. The Predator light fighter spans about 1,100 units nose to tail, which puts a unit near a centimetre (unverified: it assumes a fighter about 11 m long). A part's origin is in the model's frame, whatever its parent: object_add_part (0x004760C0) hangs every part's node from the object's root at it, and object_link_parts (0x00476130) then hangs each from its parent part's, keeping it where it is.

Unread fields¶

These are present in every record and read by nothing in the engine: the header's 0x04 scalar and the face's 0x3C word. The reader preserves them.

The mass properties at 0x68 to 0x90 place an object's origin at its parts' centre of mass (object_recentre, 0x004769F0) and give it a moment of inertia (object_bounds, 0x00476680), moved from each part's origin to the object's. Unverified: that they are integrals over the part's volume; the engine uses them as such (Live objects).

Unknown: the interpretation of trigger polygons (0x0F). The reader keeps their records as the file holds them (shp.TriggerPolygon), and the writer writes them back, but their fields are not decoded here (#11). One model carries two of them; the engine tests the player's ship against them before it descends the collision tree.

Prior art¶

The container and chunk framing here were read from the files directly. The record field semantics, the loader's search-forward rule and the chunk catalogue come from the independent analysis in Starlancer-OSS docs/shp-format.md, which traced them to the engine's own loader. Every structure offset and count in this document was re-verified against the 440 shipped models; the bounding-box frames above are a refinement, since the box does not always match the vertex extent as stored.