//Tools

Measure it on your own machine.

Two tools that grew out of the papers and stand on their own. Both install from PyPI, run on a laptop with no configuration, and are archived on Zenodo so a result made with them can be cited to the exact version.

Tools: 2
Install: pip
License: MIT
Archived: Zenodo

v0.2.3, Oct 2026

llama-roofline

Is your llama.cpp decode memory-bandwidth-bound? Measure it, in one command.

Generating a token reads every weight of the model from memory, so decode speed on a CPU is set by memory bandwidth divided by the bytes each token has to read. llama-roofline measures both sides of that on your own machine: the read bandwidth your memory actually sustains, and your own GGUF models through llama-bench across a thread sweep. It then tells you, in words, where you sit and which thread count to use. It also opens a GGUF file and reports what it actually contains under its format label.

Install and run

pip install llama-roofline
llama-roofline diagnose

Commands

llama-roofline diagnose finds your GGUF files and llama-bench, measures, and explains the result
llama-roofline diagnose ~/models --quick one model, a directory or a glob; fewer settings, one repetition
llama-roofline inspect model.gguf what the file contains under its format label, and the bytes decode reads
llama-roofline run --models ~/models/*.gguf the full sweep: every model, the fit, the figure

Release

Version: 0.2.3
Command: llama-roofline
Python: 3.9+
License: MIT

Cite it

All versions, always the newest: doi:10.5281/zenodo.21842493

Exactly v0.2.3: doi:10.5281/zenodo.23341323

The repository carries a CITATION.cff, so GitHub's "Cite this repository" button gives BibTeX and APA.

Links

v0.2.2, Oct 2026

ML Systems Lab

Reproducible ML inference benchmarking across laptops, single-board computers and servers.

One YAML config and one command run an experiment matrix across any mix of machines: locally, over SSH, or inside a Kubernetes pod. Every measurement comes back as a self-describing JSON record that carries the hardware, OS, kernel, backend version and build, the metrics, and the physical state of the machine while they were taken: power, temperature, clocks, throttle flags and the CPU quota that applied. The analysis layer turns a directory of records into tables and figures.

Install and run

pip install ml-systems-lab
mlsys probe
mlsys membw

Commands

mlsys probe what this machine can measure: power, thermals, GPU, clocks
mlsys membw its sustained DRAM read ceiling, in about a minute
mlsys run configs/example-smoke.yaml your own sweep, from a checkout of the repository
mlsys report runs/smoke-test --full tables and figures from a directory of records

Release

Version: 0.2.2
Command: mlsys
Python: 3.9+
License: MIT

Cite it

All versions, always the newest: doi:10.5281/zenodo.21867055

Exactly v0.2.2: doi:10.5281/zenodo.23329333

The repository carries a CITATION.cff, so GitHub's "Cite this repository" button gives BibTeX and APA.

Links

Next Where to hear about it: talks