Manu Nicholas Jacob / Austin, Texas
I find out what actually
limits a computer.
Hardware engineer at Dell Technologies working on enterprise AI server platforms, and an independent researcher on what really governs machine-learning inference at the edge. Every number on this site was measured on real silicon.
Edge AI /Computer architecture /PCIe /Memory bandwidth /INT8 quantization /Thermal control /Power telemetry /Root-cause analysis /ARM Cortex-A76 /Reproducible benchmarking /GPU platforms /Embedded Linux
- 13
- Papers
- 11
- Under review
- 10
- Artifact DOIs
- 1
- Patent authorized
The through-line
The memory wall, not compute, governs deep-neural-network inference on real hardware.
The same bottleneck turns up in mobile CNNs, in Vision Transformers and in on-device language models. The machine spends its time moving weights, not multiplying them, so bandwidth predicts what a network will do and FLOPs do not. Every paper below follows that one idea from a Raspberry Pi to an x86 laptop, and next to enterprise servers.
Results that changed how I build.
Three papers from the current program. Each headline number was reproduced on hardware before it was written down.
The Memory Wall Governs Edge DNN Inference
Thread scaling, multi-tenant interference and INT8 speedups are not three separate phenomena. All three fall out of a single measured bandwidth ceiling.
- Models
- 9 CNNs + 1 ViT
- Silicon
- Arm Cortex-A76, revalidated on x86
- Instrumentation
- perf counters + PMIC power
20.5% leave-one-model-out prediction error across 9 CNNs and a ViT
ACM Transactions on Embedded Computing Systems
Latency-Optimal Is Not Energy-Optimal
Running an edge SBC at maximum clock wastes roughly a fifth of the energy per inference, and the waste is core-stall energy rather than DRAM energy. That reverses the usual intuition.
~20% energy wasted at max clock, with a deployable clock-selection policy
ACM Transactions on Embedded Computing Systems
The Cold-Start Tax
The first systematic characterisation of the duty-cycled wake transient on an edge SBC. The tax is weight-loading-bound, predictable from model size alone, and falls off an eviction cliff. Includes GD-Tax, a tax-aware cache that beats LRU and LFU.
5x to 23x wake tax, R^2 = 1.00 loading-bound fit, >80x blow-up past eviction
IEEE Internet of Things Journal
13 papers total · see all
The code behind the numbers.
Every published result ships with the harness that produced it, archived on Zenodo with a citable DOI.
rpi5-quantization-benchmark
↗Controlled evaluation of INT8 quantized models on Raspberry Pi 5, isolating export format and graph optimisation from the weights themselves.
3.8x to 11.7x speedups, up to 43x better energy-delay product
edge-llm-memory-wall
↗Decode-side roofline for on-device language models, plus the KV-cache capacity wall and an energy-aware serving policy.
Decode roofline R^2 = 0.98, on Arm and on x86
edge-cold-start-tax
↗Wake-transient characterisation for duty-cycled edge inference, including GD-Tax, a tax-aware cache policy that beats LRU and LFU.
5x to 23x cold-start tax, >80x past the eviction cliff
llama-roofline
↗A command-line tool that measures where a local LLM actually sits against the memory-bandwidth roofline on your own machine. Built out of the edge-LLM work and released standalone.
The rest of the practice.
A research program needs infrastructure, and a field needs people who maintain it, review it, and judge it. All three are part of the job.
Platform
ML Systems Lab
One config file, one command, every machine on the bench. A reproducible benchmarking framework spanning laptops, Raspberry Pi and GPUs, with per-rail power, thermal and utilization telemetry in every record.
Reproduces the published roofline fits exactly; independent campaigns agree within 1.7%
DOI 10.5281/zenodo.21867055
Upstream
Fixes to the field's infrastructure
Three merged correctness fixes in NVIDIA's garak, plus open pull requests to vLLM: tokenizer edge cases, JIT preflight, benchmark robustness, harness correctness. Root-cause habits, applied upstream.
3 merged, 11 open across vLLM and garak
vllm-project · NVIDIA
Peer review
Artifact Evaluation Committee, ATC '26 (ACM SIGOPS)
Assessing submitted artifacts for functionality and reproducibility at a CORE A systems venue. One review delivered for the Journal of Open Source Software and three more assigned, plus repeat judging at hackathons and VEX Robotics events.
ATC '26 · JOSS · HackTX · VEX Robotics
Service →
Recent notes.
- 09 Aug 2026 How this site is built A static site with no CMS, no build-time magic, and a deliberate rule that the content is data rather than markup. →
- 22 Jul 2026 Reliability is a design property, not an operational outcome How well a system survives contact with reality is mostly decided at design time, not during the incident. →
- 15 Jul 2026 Constraints shape system behavior long before they break it Limits change feedback and stability well before anything crosses a threshold and trips. →