//Projects
Code, benchmarks and hardware.
A benchmarking platform, the reproducible artifact behind every paper, contributions upstream, and the hardware it all started on.
- Repositories
- 12
- Archived DOIs
- 10
- Merged upstream
- 3
- Open upstream
- 11
Platform
ML Systems Lab
One config file, one command, every machine on the bench.
A reproducible benchmarking framework for ML inference across heterogeneous hardware. One YAML config drives llama.cpp and ONNX Runtime sweeps on laptops, Raspberry Pi over SSH, and GPUs; every run produces a self-describing record carrying the hardware, backend versions, measured metrics, and the physical state of the machine while they were measured: power, temperature, clocks, throttle flags, utilization. The analysis layer turns a directory of records into publication-quality tables and figures.
Architecture
- 01 config.yaml machines, models, and the sweep matrix, declared once
- 02 RunSpecs deterministic run ids; interrupted campaigns resume by skipping what is on disk
- 03 Device local subprocess or agent pushed over SSH; a new machine is a config block, not code
- 04 Agent pure standard library, runs on the device under test; nothing crosses the network inside a measurement window
- 05 RunRecord raw output parsed on the host, so a parser bug is a re-parse, not a re-run; failures are records too
- 06 Analysis text, Markdown and LaTeX tables; roofline and scaling figures
Supported hardware
- › Raspberry Pi 5, Cortex-A76: SSH agent, PMIC per-rail power, throttle bits, DVFS control
- › i7-12700H laptop, Windows: local agent, 20-thread sweeps, models up to 7B
- › RTX 3050 via ONNX Runtime DirectML: modeled as its own device
Representative result
Reproduces the IEEE Transactions on Computers submission’s roofline fits exactly (Pi 5: 10.7 GB/s effective, R² = 0.980; i7-12700H: 35.7 GB/s, R² = 0.980), and two campaigns run natively by the framework re-measured the same quantities independently and agreed within 1.7%. Same config shows the GPU losing at batch 1 (dispatch overhead) and winning 29x at batch 64.
Related publications
- The Memory Wall at the Edge of Language
- The Memory Wall Governs Edge DNN Inference
- The Cold-Start Tax
Research artifacts
The harness, data and analysis behind each paper. Archived on Zenodo so a result can be cited and re-run.
rpi5-quantization-benchmark
↗Controlled evaluation of INT8 quantized models on Raspberry Pi 5, isolating export format and graph optimisation from the weights themselves.
3.8x to 11.7x speedups, up to 43x better energy-delay product
DOI 10.5281/zenodo.21844863
edge-llm-memory-wall
↗Decode-side roofline for on-device language models, plus the KV-cache capacity wall and an energy-aware serving policy.
Decode roofline R^2 = 0.98, on Arm and on x86
DOI 10.5281/zenodo.21844855
edge-cold-start-tax
↗Wake-transient characterisation for duty-cycled edge inference, including GD-Tax, a tax-aware cache policy that beats LRU and LFU.
5x to 23x cold-start tax, >80x past the eviction cliff
DOI 10.5281/zenodo.21844857
pi5-thermal-proxy
↗Software-only thermal proxy for commodity SBCs, released with an open 13-hour telemetry dataset intended for reuse.
Cross-validated coupling law, no added sensors
DOI 10.5281/zenodo.21844859
edge-thermal-margin-control
↗Convex thermal-margin allocation for multi-tenant edge inference, with the operating regimes where it fails reported alongside the ones where it wins.
DOI 10.5281/zenodo.21844861
latency-elastic-edge-inference
↗Model-predictive thread allocation that holds a latency service objective while the platform is under thermal and contention pressure.
~3.4x reduction in p99 latency under load
DOI 10.5281/zenodo.21844865
edge-format-tax
↗Matched-byte comparison of GGUF quantization formats on three CPU microarchitectures, with per-token energy from the Pi 5 PMIC. The label on the file turns out to be a poor predictor of what it costs to run.
Same label, same bit width, up to 38% apart in decode speed
DOI 10.5281/zenodo.21938812
edge-breakeven-speedup
↗The energy identity behind multithreaded edge inference, measured rather than assumed: 831 runs across nine architectures, four clock frequencies and one to four threads, with power integrated from the on-board PMIC.
Break-even speedup near 1.8x, sign predicted in 64 of 64 configurations
DOI 10.5281/zenodo.21987261
Tools
Things that outgrew the paper they came from and became usable on their own.
Lab and pipelines
Infrastructure that makes the measurements repeatable.
Edge-sbc-reliability-lab
↗Cross-runtime characterisation harness for machine-learning workloads on single-board computers.
edge-ai-colab-to-pi-pipeline
↗End-to-end path from training a model in Colab to running and measuring it on a Raspberry Pi, built so a result can be re-derived from scratch.
spectral-geometry-instability
↗Research pipeline treating covariance eigenspace rotation as an early risk signal.
Contributions to the infrastructure itself.
Contributions to the projects this research already depends on. Root-cause habits transfer: read the failing system, isolate the fault, fix the cause rather than the symptom.
vllm-project/vLLM
4 openThe serving engine much of production LLM inference runs on. Fixes and docs from reproducing real failures: tokenizer edge cases, FlashInfer JIT preflight, MoE tuning robustness, WSL2 gotchas.
- #50752 Encode special tokens in HF processors for transformers-native mistral-common tokenizers
- #50711 benchmark_moe: do not abort tuning when a candidate config fails Triton compilation
- #50751 Troubleshooting: FlashInfer JIT failure on a GPU newer than the local CUDA toolkit
- #50784 Troubleshooting: add a WSL2 section (pinned memory gates, lingering workers)
NVIDIA/garak
3 merged 7 openNVIDIA’s LLM vulnerability scanner. Correctness fixes found by reading the code the way a root-cause engineer reads a failing system.
- #2014 fix(detectors): honour config_root in goodside.RileyIsnt Merged
- #2011 fix: restore unreachable no-default-class error in load_plugin Merged
- #2012 fix(probes): guard badchars ASCII selection against max_ascii bounds Merged
- #2013 fix(detectors): pass response text to the Refusal judge
- #2015 probes: add HouYi prompt injection probe and detector
- #2008 probes: add email leak via virtualization latent injection
- #2009 buffs: add FormatSpread prompt-format-sensitivity buff
- #2030 harnesses: skip falsy intents instead of registering them
- #2031 analyze: count prompt characters, not dict length, in count_tokens
- #2032 fix the lite-config hint guard and its masked AttributeError
Seeing potholes coming.
A year-long, team-built in-vehicle detection system: stereo depth sensing, CNN detection on embedded hardware, a custom PCB, and a driver alert display. Shipped as a product, not a demo.
Pothole Detection System
ECE senior design · UMass Amherst · Sep 2024 to May 2025Potholes cause billions of dollars in damage a year, and drivers swerving or braking to avoid them is its own hazard. This year-long capstone detects them in real time instead: stereo cameras build a depth point cloud of the road, a CNN detects potholes from the visual and depth data on Raspberry Pi hardware, and an in-vehicle display gives the driver an immediate visual alert. The prototype hit 80 to 90% detection accuracy at speeds up to 35 to 45 mph, with ultrasonic sensors for multi-modal sensing and a custom PCB handling power, sensor interfacing and Pi connectivity. Self-contained, installable in most vehicles, and built to stay affordable.
80-90% detection accuracy, operational at 35-45 mph
Where the hardware habit started.
Award-winning weekend builds from before the research program. Kept here because everything above traces back to them.
WorldWide Rover
HackUMass XII
A rover anyone could drive from anywhere over the internet, with onboard obstacle avoidance and maze solving when nobody was steering.
4Sight
HackUMass X
A wearable that turns ultrasonic distance readings into haptic feedback, so obstacles are felt rather than heard.
Booth Beacon
HackUMass IX
RFID and distance sensors on dining-hall booths, surfaced as a live seat-availability map. First real 3D-printing and Raspberry Pi build.