Decoding REXPaint Files in Odin

REXPaint is an ASCII art tool by Kyzrati, the developer behind the fantastic roguelike Cogmind. You can use it to design UI, full-screen art, and level prefabs, which makes it a great fit for my game, Fimbul. In this post, we’ll walk through decoding REXPaint’s .xp files. If you haven’t worked with binary file formats before, this is a gentle one to start with!

Grid Sage Games publishes the official REXPaint manual as plaintext¹, and Steve Landey hosts a more readable HTML version². I’ll cite the HTML version throughout this post because its sections have direct links. Everything here is based on REXPaint v1.70.

Prerequisites

To follow along, you’ll need the Odin compiler. I’m using version dev-2026-10. The snippets below build up an xp package in its own directory, next to a main.odin that uses it:

.
├── main.odin
├── test.xp
└── xp
    ├── decode.odin
    └── xp.odin

Each snippet shows the imports it adds, but they belong at the top of the file.

Step 0: The Format

If we look at a REXPaint file with xxd, we can see that it’s gzipped:

$ xxd ./assets/screens/title.xp | head -1
00000000: 1f8b 0800 0000 0000 000b ed9c db6d c330  .............m.0

The first two bytes, 1f8b, are the magic number, a fixed byte sequence that identifies a file’s format. Every gzipped file starts with it. Next comes 08, the compression method. RFC 1952 defines only one method, 8, which is DEFLATE, and reserves 0 through 7.³ DEFLATE itself combines LZ77 and Huffman coding.

Once decompressed, the layout is simple. Every value is little-endian, and the whole file is a handful of integers followed by a long run of cells.⁴

To see those fields in a real file, we have to decompress it first. Piping the test file from later in this post through zcat gives us:

$ zcat ./test.xp | xxd | head -1
00000000: ffff ffff 0100 0000 4600 0000 0700 0000  ........F.......

Read four bytes at a time, that’s a version of -1, one layer, a width of 70 (0x46), and a height of 7.

Each layer stores its own width and height, followed by width * height cells of exactly 10 bytes each. The cells are in column-major order, meaning REXPaint writes the entire first column top to bottom, then the second column, and so on. Each cell’s character field is an index into REXPaint’s font, which uses the CP437 code page by default.⁵ That means 1 is a smiley face (☺), not a control character.

Empty cells follow one more rule. REXPaint treats a background color of 255, 0, 255 (hot pink) as transparent, and every empty cell uses it, even on the base layer.⁶ When layers are drawn on top of each other, any transparent cell still showing on the lowest layer should be drawn as black.

In Odin, each of those pieces becomes a struct:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

Color :: struct {
	r: u8,
	g: u8,
	b: u8,
}

Cell :: struct {
	character:  i32,
	foreground: Color,
	background: Color,
}

Layer :: struct {
	width:  i32,
	height: i32,
	cells:  []Cell,
}

File :: struct {
	version: i32,
	layers:  []Layer,
}

Decoding can fail in a few different ways, and we want to know which one happened. A plain bool can only tell us that something went wrong, not why, so we’ll define our own error types up front:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

import "core:compress/gzip"

Read_Error :: enum {
	None = 0,
	Offset_Goes_Past_Data, // The data ended before we finished reading
	Unable_To_Read_Value, // A value couldn't be decoded
	Bad_Dimensions, // Negative layer count, width, or height
	Size_Overflow, // A layer has more cells than MAX_CELLS_PER_LAYER
}

Decode_Error :: union #shared_nil {
	gzip.Error,
	Read_Error,
}

Read_Error covers everything our own parsing can get wrong. Decode_Error wraps it together with gzip’s error type, letting a single return value describe either kind of failure. The #shared_nil directive means a nil from any variant (including Read_Error.None) makes the whole union nil, and err == nil reads as “everything worked.” The procs that parse bytes will return a Read_Error, and only the top-level procs return the full Decode_Error.

Step 1: Inflating with core:compress/gzip

Odin’s core:compress/gzip⁷ package can decompress a file into a byte buffer for us. We’ll create a procedure (Odin’s name for a function, or “proc” for short) called decode_from_file, and stub out decode_from_bytes for later:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

package xp

import "core:bytes"
import "core:compress/gzip"

decode_from_file :: proc(
	path: string,
	allocator := context.allocator,
) -> (
	f: File,
	err: Decode_Error,
) {
	buf: bytes.Buffer
	defer bytes.buffer_destroy(&buf)
	gzip.load_from_file(path, &buf, allocator = allocator) or_return

	raw := bytes.buffer_to_bytes(&buf)

	return decode_from_bytes(raw, allocator)
}

decode_from_bytes :: proc(
	b: []u8,
	allocator := context.allocator,
) -> (
	f: File,
	err: Decode_Error,
) {
	// TODO: read version and layers...
	return {}, nil
}

decode_from_file does the following:

1. Declares an empty bytes.Buffer⁸ struct called buf
2. Defers cleanup of buf until the end of the scope
3. Loads and decompresses the file at path into buf using allocator
4. Gets the decompressed bytes out of buf with bytes.buffer_to_bytes
5. Passes those bytes to the stubbed decode_from_bytes and returns its result

The gzip line uses or_return, which is Odin’s shorthand for the most common error-handling pattern. If the last value a call returns is a non-nil error, or_return returns that error from the current proc right away. Because gzip.Error is one of the variants of Decode_Error, gzip’s error passes straight through to our caller. Without or_return, that line would look like this:

1
2
3
4

// Roughly the same as: gzip.load_from_file(path, &buf, allocator = allocator) or_return
if load_err := gzip.load_from_file(path, &buf, allocator = allocator); load_err != nil {
	return {}, load_err
}

You might notice that raw points into buf, which the defer frees. Freeing it is safe here, because the return expression runs before any deferred statement, and decode_from_bytes copies every value it needs into the File. We’ll fill in that stub in just a bit!

At this point, we have the raw bytes of the decompressed .xp file. Turning them into a File struct means reading fixed-size chunks: 4 bytes for an i32, 1 byte for a u8.

Step 2: A Tiny Cursor

A parser only needs to know two things, the bytes and where it is in them, and a two-field struct covers both.

1
2
3
4

Reader :: struct {
	raw:    []u8,
	offset: int,
}

On top of Reader, we need two helpers: one that reads an i32 and one that reads a u8. Each one returns the value and moves offset forward.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

import "core:encoding/endian"

read_i32 :: proc(r: ^Reader) -> (v: i32, err: Read_Error) {
	if r.offset + 4 > len(r.raw) {
		return 0, .Offset_Goes_Past_Data
	}

	val, ok := endian.get_i32(r.raw[r.offset:], .Little)
	if !ok {
		return 0, .Unable_To_Read_Value
	}
	r.offset += 4

	return val, .None
}

read_u8 :: proc(r: ^Reader) -> (v: u8, err: Read_Error) {
	if r.offset >= len(r.raw) {
		return 0, .Offset_Goes_Past_Data
	}

	val := r.raw[r.offset]
	r.offset += 1

	return val, .None
}

read_i32 leans on the core:encoding/endian package⁹ to decode 4 little-endian bytes. endian.get_i32 does its own length check and returns false when fewer than 4 bytes are left, but it can’t tell us why it failed. Checking the length ourselves first lets us report .Offset_Goes_Past_Data, the error a truncated file will actually hit. read_u8 indexes the slice directly, and without its check, reading past the end would panic.

Step 3: Parsing Layers and Cells

The first field we need to parse is version. Our new read_i32 proc handles that, and then we hand the rest of the bytes off to read_layers:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

decode_from_bytes :: proc(
	b: []u8,
	allocator := context.allocator,
) -> (
	f: File,
	err: Decode_Error,
) {
	r := Reader {
		raw    = b,
		offset = 0,
	}

	f.version = read_i32(&r) or_return
	f.layers = read_layers(&r, allocator) or_return

	return f, nil
}

read_i32 returns a Read_Error, but decode_from_bytes returns a Decode_Error. or_return still works here because Read_Error is one of the union’s variants.

read_layers is where things get more interesting:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

import "base:runtime"

// A layer header consists of a width and height. i32 + i32 = 4 bytes + 4 bytes = 8 bytes
LAYER_HEADER_BYTES :: 8

read_layers :: proc(r: ^Reader, allocator: runtime.Allocator) -> (l: []Layer, err: Read_Error) {
	count := read_i32(r) or_return
	if count < 0 {
		return nil, .Bad_Dimensions
	}

	// Every layer needs at least its header, so a count the remaining bytes
	// can't hold is bogus. Reject it before allocating.
	if i64(count) * LAYER_HEADER_BYTES > i64(len(r.raw) - r.offset) {
		return nil, .Offset_Goes_Past_Data
	}

	l = make([]Layer, int(count), allocator = allocator)
	// Clean up our mess if there was an error.
	defer if err != .None {
		for layer in l {
			delete(layer.cells, allocator)
		}

		delete(l, allocator)
	}

	for i in 0 ..< int(count) {
		l[i] = read_layer(r, allocator) or_return
	}

	return l, .None
}

There are two ideas here worth slowing down for.

The first is that we never trust a number from the file until we’ve checked it against the data we actually have. The layer count is just 4 bytes that someone else wrote. If a file is truncated or corrupted, it could claim to have two billion layers, and a naive make would try to allocate all of them. Each layer needs at least an 8-byte header. Any count that couldn’t possibly fit in the remaining bytes gets rejected before we allocate anything. We also widen to i64 before multiplying, because a large count * 8 can overflow an i32.

The second is defer if. Odin’s defer runs a statement when the scope exits, and defer if only runs it if the condition is true at that moment. Here the condition is err != .None, so the block only runs on failure. If any read_layer call fails partway through, or_return sets err to that error and returns, and the deferred block frees every layer we already read. On success, err is .None and the caller owns the memory.

read_layer follows the same pattern:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

// A sanity limit, so a corrupt header can't ask for a huge layer.
MAX_CELLS_PER_LAYER :: 4096 * 4096

// A cell consists of a character, a foreground color, and a background color.
// i32 + 3 × u8 + 3 × u8 = 4 bytes + 3 bytes + 3 bytes = 10 bytes
CELL_BYTES :: 10

read_layer :: proc(r: ^Reader, allocator: runtime.Allocator) -> (l: Layer, err: Read_Error) {
	width := read_i32(r) or_return
	height := read_i32(r) or_return

	if width < 0 || height < 0 {
		return {}, .Bad_Dimensions
	}
	l.width, l.height = width, height

	n := i64(l.width) * i64(l.height)
	if n > MAX_CELLS_PER_LAYER {
		return {}, .Size_Overflow
	}

	// Same idea as the layer count: don't allocate for cells that aren't there.
	if n * CELL_BYTES > i64(len(r.raw) - r.offset) {
		return {}, .Offset_Goes_Past_Data
	}

	l.cells = make([]Cell, int(n), allocator = allocator)
	defer if err != .None {
		delete(l.cells, allocator)
	}

	for i in 0 ..< int(n) {
		l.cells[i] = read_cell(r) or_return
	}

	return l, .None
}

Notice that a failed read_layer cleans up its own cells. Because or_return exits before l[i] is assigned in read_layers, the failed layer never makes it into the slice. Nothing gets freed twice.

Finally, cells and colors are just a few reads in a row:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

read_cell :: proc(r: ^Reader) -> (c: Cell, err: Read_Error) {
	c.character = read_i32(r) or_return
	c.foreground = read_color(r) or_return
	c.background = read_color(r) or_return

	return c, .None
}

read_color :: proc(r: ^Reader) -> (c: Color, err: Read_Error) {
	c.r = read_u8(r) or_return
	c.g = read_u8(r) or_return
	c.b = read_u8(r) or_return

	return c, .None
}

Remember that cells are stored in column-major order. The cell at (x, y) lives at index x * height + y, not y * width + x. A small helper keeps that detail in one place:

1
2
3

cell_at :: proc(l: Layer, x, y: i32) -> Cell {
	return l.cells[x * l.height + y]
}

Step 4: Putting It to Use

decode_from_bytes hands back a File with nested allocations, since every layer in its slice owns a slice of cells. Whoever calls it owns all of that memory and needs a way to free it with the same allocator that decoded it:

1
2
3
4
5
6
7

file_destroy :: proc(f: File, allocator := context.allocator) {
	for layer in f.layers {
		delete(layer.cells, allocator)
	}

	delete(f.layers, allocator)
}

Now we can decode a real file and print its first layer to the terminal:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

package main

import "core:fmt"
import "xp"

main :: proc() {
	f, err := xp.decode_from_file("test.xp")
	if err != nil {
		fmt.eprintln("failed to decode test.xp:", err)
		return
	}
	defer xp.file_destroy(f)

	if len(f.layers) == 0 {
		fmt.eprintln("test.xp has no layers")
		return
	}

	layer := f.layers[0]
	for y in 0 ..< layer.height {
		for x in 0 ..< layer.width {
			c := xp.cell_at(layer, x, y)
			// CP437 and ASCII only agree on the printable range.
			if c.character >= 32 && c.character <= 126 {
				fmt.print(rune(c.character))
			} else {
				fmt.print(' ')
			}
		}
		fmt.println()
	}
}

If decoding fails, fmt prints the variant inside the union, so a truncated file reports Offset_Goes_Past_Data instead of a bare “it didn’t work.” A caller that needs to react differently could switch on err instead. Running it against the small test file, where each letter is drawn out of copies of itself, gives us:

 H   H EEEEE L     L      OOO         W   W  OOO  RRRR  L     DDDD  !
 H   H E     L     L     O   O        W   W O   O R   R L     D   D !
 HHHHH EEEE  L     L     O   O        W W W O   O RRRR  L     D   D !
 H   H E     L     L     O   O  ,     WW WW O   O R  R  L     D   D
 H   H EEEEE LLLLL LLLLL  OOO  ,      W   W  OOO  R   R LLLLL DDDD  !

Because the terminal prints row by row, we loop y on the outside and x on the inside. cell_at takes care of the column-major storage underneath. Glyphs outside the printable ASCII range need a CP437-to-Unicode lookup table to display correctly, but this is enough to see our screen show up.

Wrap-Up

In about 150 lines of Odin, we took a binary file, decompressed it, and mapped its bytes into a structure our program can use anywhere. Going the other way and writing .xp files from a File struct is mostly the same code in reverse, though we’d need something outside Odin’s standard library to handle the compression. If you want to see the whole thing in one place, the full source is on Codeberg: decode.odin and xp.odin.

In a future post, I’d like to go over how I’m using .xp files in Fimbul for title, win, and death screens, as well as prefabs that spawn enemies and items. In the meantime, try running a .xp file through zcat and xxd and see how much of it you can read by hand now!

Until next time 🤙