vunreleased (no tag yet) · Apache-2.0 · GitHub
mojo-baro runs open-weight language models on one AMD Radeon RX 7900 XTX. It reads a GGUF file, builds an engine for that model's shape, and serves chat over HTTP. Every GPU kernel in it is written in Mojo.
The RX 7900 XTX has 24 GB of VRAM, more than most consumer cards. The vendor libraries for decode and matrix multiplication are not tuned for these cards, so the question is how much speed they leave out. mojo-baro measures that gap. Each result in the repository has a preregistered protocol in bench/, a check you can run, and a date. Rows measured on an older toolchain say so.

Three parts. baro-serve is a Rust HTTP front. It handles the chat template, the tokenizer, tool calls, JSON-schema masks, the request queue, cancellation, per-message checkpoints and /health. It talks over a line protocol (serve/PROTOCOL.md) to an engine process written in Mojo.
The engine is compiled for one model shape. Layer count, hidden size and head counts are compile-time constants, so the loops have fixed trip counts and the compiler can unroll them. The cost is that an engine built for a 9B pack refuses a 27B pack.
Weights are not read from GGUF at run time. tools/baro converts the GGUF once into a pack: one flat pack.bin with weights laid out for the kernels that read them, an index, the tokenizer, and an identity file that names the source by sha256. A pack has q4 or q8 weights (requantised), native K-quant weights (the GGUF's Q4_K and Q6_K blocks, decoded inside the kernel), or split packs for MoE experts held in host RAM.
Every response reports which decode path served it. mega runs one persistent kernel per token for the whole forward pass, the dense default. launch runs one kernel per operation, the reference path and the MoE default. spec is speculative decoding with the model's own MTP draft head, and its output is identical to greedy decoding. prefill runs the prompt in batched chunks through WMMA matrix multiplies.
KV cache is f32 by default, or int8 with BARO_KVQ=int8 on dense models: 33,280 bytes per token on the 9B.
Command line, through tools/baro:
serve MODEL.gguf [--port N]: builds the engine and the pack for the file's shape the first time, which takes a few minutes, caches both under ~/.cache/baro/, and serves. Later runs of the same file start in seconds.chat: a terminal client. It builds baro-chat on first use.gui [--model MODEL.gguf] [--port N] [--fps N]: one desktop window with a streaming chat pane, an engine panel, settings tabs, presets, and an Apply step that restarts the engine with new settings. The frame cap defaults to 30 fps.pack-path MODEL.gguf: prints the cached pack path and says PACK FOUND or PACK MISS. It never builds.receipts MODEL.gguf: lists the GGUF receipt for the file.verify: checks an engine file's embedded harness against git at the commit it names.run MODEL.gguf: runs a file from the harness it carries, without the source checkout.eval MODEL-KEY [--quick N] [--dry-run]: a quality run for one key in bench/quality-models.json.eval-all: the quality sweep, bench/quality-sweep.sh.multi-serve MODEL.gguf --devices 0,1: starts one baro-serve node per GPU. baro-router puts the nodes behind one port and routes by model and load.swarm: runs tools/gpu_512_persona_swarm.mojo.HTTP API, from baro-serve:
POST /v1/chat/completions, the OpenAI shape, with stream: true for server-sent events.POST /v1/completions, GET /v1/models, POST /v1/cancel and POST /v1/fork, which copies a session's state to a second engine.POST /tokenize and /detokenize, which match llama.cpp's output on the same GGUF.GET /health, which returns 200 only when an engine can take work, and GET /props for the engine limits./api/chat, /api/generate, /api/tags, /api/ps, /api/show and /api/version.POST /v1/embeddings and /api/embeddings, using bge-small-en-v1.5 (384 dimensions) by default.
Measured on one RX 7900 XTX. Decode is greedy, with no speculation on either side unless the row says so.
| model, weights | llama.cpp tok/s | mojo-baro tok/s | ratio | date |
|---|---|---|---|---|
| Qwythos-9B, Q4_0 both sides, 20-prompt median | 111.34 | 148.89 | 1.34x | 2026-10-12 |
| Qwythos-9B, Q8_0 both sides, same stint | 75.46 | 80.38 | 1.07x | 2026-10-26 |
| Ornith-1.5-9B Q4_K_M, native K-quant pack, megakernel | 94.14 | 99.53 | 1.06x | 2026-10-26 |
You need an RDNA3 card (gfx1100, 24 GB), Linux with ROCm 7.2, uv, Rust, CMake with a C++ compiler, and jq and curl. 32 GB of RAM is enough for 9B models. No weights ship with the repository.
git clone https://github.com/amarbarosys/mojo-baro && cd mojo-baro
uv sync
(cd serve && cargo build --release)
bench/preflight.sh
tools/baro serve ~/Models/Qwen2.5-7B-Instruct-Q4_K_M.gguf --port 8080
curl -s localhost:8080/v1/chat/completions -H 'content-type: application/json' -d '{
"messages": [{"role": "user", "content": "Name the capital of France in one word."}],
"max_tokens": 16000
}' | jq -r '.choices[0].message.content'
bench/preflight.sh needs no GPU and no weights. uv sync pins max[all]==26.7.0, which is Mojo 1.2.0. Any OpenAI client works with base_url="http://localhost:8080/v1".
BARO_MEGA=0 selects the launch path.Source: https://github.com/amarbarosys/mojo-baro (public, Apache-2.0). Protocols, receipts and the full results are in bench/ and docs/.