Frogger File Information Megathread

Hello,

I spent a good deal of time reverse engineering Frogger file formats, and developing FrogLord.
I figured I could condense all the information scattered around about file formats, and try to condense it into a single thread.
If you're interested in having an API which can read / write these files, check out FrogLord.

NOTE: This guide is currently unfinished. It will be updated further.
Last Changed: 10 / 7 / 2019.


[STR|PSX] Video + Music Format:
These files are just normal ol' Sony PS1 STR files. The tool jPSXdec can view and export them.

NOTE: jPSXdec does not rip "proper" STR files, I had to use some obscure French tool to rip "proper" STR files. This was only relevant to me, for running a compiled version of Frogger. I do not believe this information is relevant to anyone else, but I am putting it here just in case.

[RPL|PC] Video Format (CD):
These files appear to be the same video file format used by Tomb Raider. That's also why there's an Eidos logo on the PC version.
There are converters and viewers for this file format, but no known way to create these files.

Parser Source Code: here
RPL Video Converters: Online
RPL Desktop Video Players: FinalMedia Player, RPL Player, Mirror RPL Player

[MWD,MWAD] Medievil WAD (FROGPSX.MWD):
This is the main game archive. It contains nearly all game files.

To get files out of the MWD, there are two options.
1. Use FrogLord. Ctrl + O or File > Output Frogger File to export the file you want. [This will save files individually]
2. Download QuickBMS, open this text file, and use Ctrl + S to save it as 'frogger.bms'. Then run QuickBMS, and use 'frogger.bms' to get all of the files in the MWD. [This will save the raw files from the MWD in bulk]

struct MWDFile {
 fixed length string "DAWM"; // MWAD
 4 null bytes;
 // Can put some strings here, the vanilla MWD has its creation time here.

 // Everything beyond here is a game file. Every game file always starts at a multiple of 0x800. If there are unused bytes between files, they are null.
 // Information about each file is kept in the MWI, as seen below.
}
PP20 Compression:

PP20 Compression is the type of compression used.
It most closely resembles LzSS.
This is an obscure compression algorithm designed in the early 90s by Nico François.

FrogLord's Unpacker: Source Code
Unpacking is a very simple process, and there was some good documentation online.

FrogLord's Packer: Source Code
Repacking proves to be more difficult.
My packer isn't perfect yet. At high compression settings, it's slow (bad time complexity), and doesn't give quite as good results as the original packer.
Its results are acceptable right now, so I'm not working on it, but I'd like to improve it in the future.

I'm still trying to figure out what technique he used to determine the next longest substring.

So, what are the options for improving?
 - Contact Nico and ask if he'll release the code to the packer or maybe some information about how it encodes.
 - Try to find a version of the packer library which we can either somehow find a way to interface with from Java, or disassemble.
 - More research, possibly talking to people experienced in compression. StackOverflow? College professors?

[MWI] Medievil WAD Index (In frogger.exe):
The MWI contains information about every file in the MWD. It is located in the Frogger executable.
It also contains file entries for files inside of WADs.

struct MR_RESINFO {
 string pointer filename; // This is always -1 on PSX, but has the filename on Windows.
 ulong flags; // See below.
 ulong filetype; // File Type ID. This identifies what kind of file this is, however it's a little... unreliable. I still don't fully understand some of its edge-cases.
 ulong sectorOffset; // sectorOffset * 0x800 = The address in the MWD that the file is located at.
 ulong pointer address; // Run-time only, address resource is loaded at. NULL = not loaded, which in the case of file format is always.
 ulong pointer unpackedAddress; // Same as above, run-time only.
 ulong filesize; // The raw file-size. If the file is compressed, this will be the size of the compressed data.
 ulong realsize; // The real file-size. This will be the file size after decompression, if it's compressed.
}
Flags:
 1 << 0: Single Access     If the file is not in a WAD.
 1 << 1: Group Access     If the file is in a WAD.
 1 << 2: Is Group            Is this a WAD file?
 1 << 3: Absolute Path     Unused. This was for a debug setting which allows loading from the hard-drive. It still may be functional in the retail or prototype.
 1 << 4: Auto Unpack      Unpack the file automatically.
 1 << 5: Manual Unpack  Don't automatically decompress the file, Frogger will handle it specially.

[Windows-Only]
After all of the info entries, the file names + file paths for each of the file entries are listed, as null terminated strings, with some slight offset padding afterwards.
The filename pointers point into this list.

[VLO] VRAM Texture Bank:
These files hold all the images in the game.

The PC and PSX file formats have a fair number of differences, which should be explained in the outline.

struct VLOFile {
 byte[4] signature; // PC is "2GRP", PSX is "2GRV". ("2GRV" -> "VRG2" -> "Vorg2". "2GRP" -> "PRG2" -> "PC Vorg 2". Vorg was the name of the program that managed textures.)
 int imageCount; // The number of images kept in this VLO file.
 GameImage* textures; // The pointer to texture data.
 
 <IF PLAYSTATION>
 int clutCount; // How many color lookup tables (CLUTs)
 Clut* cluts; // The pointer to color lookup tables. (CLUTs)
 </IF>
}

struct GameImage {
 short vramX; // The X position to place this image in VRAM. (Check out FrogLord for a visualization.)
 short vramY; // The Y position to place this image in VRAM. (Check out FrogLord for a visualization.)
 short vramWidth; // The width of this image. This should be multiplied by the width multiplier, to get the full image width.
 short vramHeight; // The height of this image.
 void* imageData; // See below for an explanation on image data.
 short textureId; // The texture id, models reference this.
 short texturePage; // This contains two bits of data, the clut mode, and the ABR. clutMode = (texturePage & 0b110000000) >> 7; abr = (texturePage & 0b1100000) >> 5; CLUT Modes: 0 - 4 Bit Clut. 1 - 8 Bit CLUT. 2 - 15-Bit No CLUT (Direct Color).

 <IF PLAYSTATION>
   short clutId;
   short flags;
 <ELSE IF PC>
   short flags;
   short clutId; // Always 0 on PC?
 </IF>

 ubyte u;
 ubyte v;
 ubyte ingameWidth; // Only some of the image will get shown in-game. Dictate how much.
 ubyte ingameHeight;

 // imageData - This data is special.
 // It is raw byte data which when read will give image pixel data.


 // If the VLO is a PC VLO,
 // 1 Pixel = 4 Bytes.
 // Raw ABGR Format.

 // Otherwise, it's a PSX VLO. Each CLUT Mode has different handling.

 // Mode 0 (4-Bit CLUT):
 //  1 Pixel = Half of a byte.
 //  The read value should then be used to get a color from the CLUT, and then that color for the current pixel.

 // Mode 1 (8-Bit CLUT):
 //  1 Pixel = 1 Byte.
 //  The read value should then be used to get a color from the CLUT, and then that color for the current pixel.

 // Mode 2 (15-Bit Color No CLUT):
 //  1 Pixel = 2 bytes. Follows the `ClutColor` format detailed below.
 //  The read value is directly the color to use for the current pixel.
}

// ================================
// BEYOND HERE IS PLAYSTATION ONLY.
// ================================

// Clut stands for color lookup table.
// A single color is made of four values, R, G, B, A. Each of these takes up a single byte, so one pixel is 4 bytes, or 32 bits. This is true on PC.
// On the PSX, a CLUT is used to reduce the amount of space a single pixel takes. In a 4-bit clut mode (the most common mode), a single pixel takes 4 bits, or half of a byte.
// This is because instead of storing the raw color data, an index is stored into this CLUT. This means an image can only use as many colors as the CLUT can hold.
// In 4-bit CLUT mode, the clut can hold 16 colors, since 15 is the max number that can be help in 4 bits (0b1111 -> 15).
struct Clut {
 short vramX; // The X position that this will be placed in VRAM.
 short vramY; // The Y position that this will be placed in VRAM. The x + y position should never overlap with another clut or another image in VRAM.
 short width; // The width of the clut.
 short height; // The height of the clut.
 ClutColor* colors; // The pointer to the clut color data. The number of color to read is (width * height).
}

// But then the Playstation takes it even further.
// It doesn't have the full RGB color range.
// Instead, it stores a full color in a short.
// RGB 5-5-5-1.
struct ClutColor {
 private short colorData;
 // Bits 0 - 4   The red color. To get this as a normal RGB888 value, bitshift this value left by 3.
 // Bits 5 - 9   The blue color. To get this as a normal RGB888 value, bitshift this value left by 3.
 // Bits 10 - 14 The green color. To get this as a normal RGB888 value, bitshift this value left by 3.
 // Bit 15       If the color is semi-transparent or not.
}
[MOF] Model Object Format (3D Models):
MOF is believed to stand for "Model Object Format". MOF files come with extensions like XAR, XMR, 
MOF files are found inside WAD files.
Each one represents a 3D model, or is dummied out, in which-case the file is just 'DUMY'.

This file format is VERY complicated, and has a LOT of unused pieces. The information below is a lot simpler than the actual data / process.
I am only giving an overview of the files here, for specific structs, data layout, and other details, checkout FrogLord's code.

There are three main "types" of MOF files:

Dummy MOF:
This is basically a blank file, with four bytes: "DUMY".
It has no data in it, and it means the file is not used, but still included, presumably to keep ids consistent between levels.

Static MOF:
A static MOF is the basic 3D model.

It contains a bunch of "MOF Parts", which are all of the different pieces of the model.
Most models only use one part, but more complicated models use a lot. For instance, Frogger's model will have a part for a leg, an arm, the frog's back, etc.

There is a special state, or "Incomplete MOF", which is just used for the checkpoints and baby frogs. (Models which are exactly the same except for color)
Lots of data is omitted from these models, and gets copied when the model is loaded, from the base model. ( frog, checkpoint flag).
These should be handled as a special-case, refer to FrogLord's code to see how to handle it. Only a few MOFs are incomplete.

Each MOF Part contains these components:
 - Polygons (The 3D geometry that makes up the part)
 - Flags (Any toggles / flags set for this specific flag, like maybe it is hidden by default or something.)
 - Partcels - Contains 3D vertex and normal data
 - Hilites - Contains lighting data.
 - Matrix - A position of some kind. Unknown what exactly this is used for, it might be origin?
 - Collprim - Collision data.
 - Animation Data (See below.) (Type of Data: Flipbook, part poly anim data)

You may be wondering, why is there animation data if this isn't an "Animated MOF".
Well, there are different types of animations.

There are two types of animations used in static MOFs:

1. Flip-book Animations (Model changes)
What that means is that each vertex's position is kept for each frame of animation, and each frame it uses the vertex positions for the frame.
This resembles how real flip-books work, which is why it is named after them.

Examples of this include:
 - The vampire bat's flying animation in cave zones.
 - The lizard's walking animation in the desert zone.
 - The baby frogs jumping and occasionally kicking up their back legs.

2. Part Poly Animation (Texture changes)
Each 3D face with an animated texture has a list of the textures to cycle through, and how many frames it will use each texture for.

Examples of this include:
 - The electric eel in the jungle has an animation which makes the eel look like it's sparking, by using this type.
 - The frog's shine.
 - The sky going across the windshield of the vehicles in the retro levels.

Animated MOF:
An animated MOF is built for doing animations based on moving "mof parts", instead of changing them.

An animated MOF contains these components:
 - A static MOF. (Yes, the other type of MOF.) This contains the basic model, which this MOF expands.
 - Transforms - A list of transformations, which can be performed on a part. This makes each part show up in the right place, with the right rotation. Check out FrogLord to see how these calculations are performed.
 - List of Animation Cels - Each animation cel is a list of indices to the list of transforms detailed above. The list of transform indices has an index for every part in every frame of animation.

Examples of these include:
 - Basically all of Frogger's animations.
 - The cave spider's walking animation. (The whole body is a single part, but each leg has its own part, so the legs can be controlled individually.)
 - All of the dog animations in the Suburbia maps (ie Bow Wow Falls)


[MAP] Map File:
This file contains nearly all the information about a given level, including entity placement, 3d geometry, animation data, collision info, etc.
Technical information on how the file is structured should probably be obtained by reading the code here.

Grid:
The grid is a two-dimensional grid which represents the all map tiles Frogger can interact with.
Each unit in the grid is a "grid stack". This grid stack contains all of the "grid squares" for a particular position.
Each "grid square" represents a single 3D tile that the frog can interact with.

This creates a very simple movement system. If you want to move left, get the grid square to the left of the current one, and handle accordingly. (Ie: If you can hop there, hop there. If it's water, hop there and kill the player for going in water)

Lights:
This contains data for how the map and entities should be lit.

Paths:
Most entities that move will follow a path.
Each path can consist of any number of path segments.

Types of path segments:
Line - The simplest segment, this has a start position and an end position.
Arc -  This segment will have the entity follow a circular arc.
Spline - This segment uses max to make funky curves. This is mainly used for erratic paths like butterfly paths in Mower Mania.

Camera Zones:
Camera zones control the camera's view on Frogger, at a given position on the map.
Each camera zone covers an area of grid squares which it will be applied in.

Entities:
Entities can be things like trucks or logs, but they can also be invisible things like sound generators or such.
Each entity type has its own entity data, which can be explored in FrogLord code.
Some entities exist at a world position, others exist on a path that they move on.
Some have 3D models, some are 2D sprites, some do not display at all.

Forms:
Forms are basically the same as grid squares, but for walking on entities, or colliding with entities.
They control where you can walk, what parts of an entity kill you, etc, just like grid squares.

Map Animations:
These are animations applied to map geometry.
The animation can be a texture animation (texture changes), a UV animation (texture scrolls), or both.


SKY_LAND.MAP:
This file is not actually a map file, but rather it is texture information for the sky maps. It's information about the texture far below representing the ground.

struct SkyLandFile {
 ushort xLength;
 ushort zLength;
 short[xLength * zLength] skyData;
}

[VB, VH] Sound Bank:
This file contains sound effects. It differs on PC vs PS1.

PC Version:

struct VHEntry {
 int hasAudio; // 0 or 1.
 int dataStartOffset; // The offset into the body where the data starts.
 int dataSize; // The number of bytes the entry takes up.
 int unknown1; // Always 1.
 int unknown2; // Always 1.
 int sampleRate;
 int bitWidth;
}
A VB File is just a bunch of headerless WAV audio chunks appended one after the other. In the PC prototype, it does have WAV headers.
The VH file is just 4-byte integer "vhEntryCount", then all of the VH entries one after the other.
The number of VH entries is the same in all VB files. This is because there is an entry for every sound in the game, even sounds which are not present in the associated VB file.
Audio extraction is as simple as getting the raw audio data from the VB files with the VH header data.

PS1 Version:
This file format is not yet fully understood by us.
A program named "PSounds" can export the sounds, but it is closed source.

Useful Resources:
Zophar Info
Xentax Post
Zenhax Post
VgmTrans Code PSXSPU.cpp
VgmTrans Code PSXSPU.h
VgmTrans Code Vab.cpp
VgmTrans Code Vab.h

[TODO] This section is incomplete.

[DAT] Demo Sequence:
If you idle on the main menu for a while, a "demo" will play, where a level loads and Frogger starts hopping around without your control.
This file contains the sequence of key-strokes played back during one demos.

struct DemoFile {
 int frameCount;
 int startXTile;
 int startZTile;
 byte[1800] actions; // 1800 = 30 * 60. So, 30 actions per frame.
}
Each "action", is a single byte.
It is comprised of a single directional input, and bit flags.

Directional Input Values (Takes up bits 0 and 1):
Up: 0x00
Right: 0x01
Down: 0x02
Left: 0x03

Bit Flags (Takes up bits 2-7):
2 = Unknown. (Probably Croak)
3 = Super Hop
4 = Tongue
5 = Rotate Counter-Clockwise
6 = Rotate Clockwise
7 = Do not hop in any direction. (Value should then be 0x00)

These flags and directional value when put together create a single action byte.

[PAL|PC] Palette:
This file format seems to be standardized. I was unable to find a full spec for the file format, but it seems to be used in other games like Worms, which even has an editor tool.
It's odd, because it seems to be completely unused.

struct PalFile {
 byte[4] signature; // "RIFF"
 int fileSize;
 byte[4] palSignature; // "PAL "
 byte[4] dataSignature; // "data"
 int dataChunkSize;
 short palVersion; // 0x300
 short colorCount; // The number of colors in the pallette.
 Color[colorCount] colors; // All of the colors in the file.
 int padding; // Always 0x00 [Probably an EOF marker]
}

struct Color {
 byte red;
 byte green;
 byte blue;
 byte flags; // Seems to always be 0x00.
}
[WAD] Holds Multiple Files:
This file holds multiple files. On the PC version, it is only used for MOFs, but the PS1 version uses it for VLOs too.

struct WadFile {
 WadEntry[] entries;
 int terminator = -1;
}

struct WadEntry {
 int resourceId; // The MWI entry ID associated with this entry.
 int fileType; // Could probably be anything, but Frogger only seems to use this for VLO and MOF files.
 int dataSize; // The size of the raw entry data.
 int padding;
 byte[dataSize] fileData; // The raw entry data. This data is used differently based on what the fileType value is. If this is an image, it is a VLO struct. If this is a model, it's MOF data.
}