Manu Nicholas Jacob / Austin, Texas
I find out what actually
limits a computer.
Hardware engineer at Dell Technologies working on enterprise AI server platforms, and an independent researcher on what really governs machine-learning inference at the edge. Every number on this site was measured on real silicon.
Edge AI /Computer architecture /PCIe /Memory bandwidth /INT8 quantization /Thermal control /Power telemetry /Root-cause analysis /ARM Cortex-A76 /Reproducible benchmarking /GPU platforms /Embedded Linux
- 13
- Papers
- 11
- Under review
- 15
- Artifact DOIs
- 1
- Invention authorized for U.S. patent filing
The through-line
The memory wall, not compute, governs deep-neural-network inference on real hardware.
The same bottleneck turns up in mobile CNNs, in Vision Transformers and in on-device language models. The machine spends its time moving weights, not multiplying them, so bandwidth predicts what a network will do and FLOPs do not. Every paper below follows that one idea from a Raspberry Pi to an x86 laptop, and next to enterprise servers.
Results that changed how I build.
Three papers from the current program. Each headline number was reproduced on hardware before it was written down.
The Memory Wall Governs Edge DNN Inference
Thread scaling, multi-tenant interference and INT8 speedups are not three separate phenomena. All three fall out of a single measured bandwidth ceiling.
- Models
- 9 CNNs + 1 ViT
- Silicon
- Arm Cortex-A76, revalidated on x86
- Instrumentation
- perf counters + PMIC power
20.5% leave-one-model-out prediction error across 9 CNNs and a ViT
ACM Transactions on Embedded Computing Systems (under review)
Latency-Optimal Is Not Energy-Optimal
Running an edge SBC at maximum clock wastes roughly a fifth of the energy per inference, and the waste is core-stall energy rather than DRAM energy. That reverses the usual intuition.
~20% energy wasted at max clock, with a deployable clock-selection policy
Sustainable Computing: Informatics and Systems (under review)
The Cold-Start Tax
The first systematic characterisation of the duty-cycled wake transient on an edge SBC. The tax is weight-loading-bound, predictable from model size alone, and falls off an eviction cliff. Includes GD-Tax, a tax-aware cache that beats LRU and LFU.
5x to 23x wake tax, R^2 = 1.00 loading-bound fit, >80x blow-up past eviction
IEEE Internet of Things Journal (under review)
13 papers total · see all
Reviewing
The other side of the desk.
Evaluating other people's work is the fastest way to stay honest about your own. This year that became a record of its own.
- Artifact evaluation, ATC '26 On the Artifact Evaluation Committee of the ACM SIGOPS continuation of USENIX ATC, rated CORE A. Both assigned artifact reviews delivered inside the window.
- Journal of Open Source Software Six reviewer assignments from six different editors. Two reviews delivered, one on a paper now published; four more in progress.
- Judging HackTX at UT Austin, and repeat judge and judge advisor at VEX Robotics V5 events.
The code behind the numbers.
Every published result ships with the harness that produced it, archived on Zenodo with a citable DOI.
qgemv-roofline
↗A quantized GEMV kernel ladder measured against its own bandwidth roof: 12 CUDA kernels plus Triton for batch-1 LLM decode, on five NVIDIA GPUs from sm_60 to sm_89.
edge-llm-memory-wall
↗Decode-side roofline for on-device language models, plus the KV-cache capacity wall and an energy-aware serving policy.
Decode roofline R^2 = 0.98, on Arm and on x86
edge-cold-start-tax
↗Wake-transient characterisation for duty-cycled edge inference, including GD-Tax, a tax-aware cache policy that beats LRU and LFU.
5x to 23x cold-start tax, >80x past the eviction cliff
llama-roofline
↗A command-line tool that measures where a local LLM actually sits against the memory-bandwidth roofline on your own machine. Built out of the edge-LLM work; live on PyPI as pip install llama-roofline.
The rest of the practice.
A research program needs infrastructure, and a field needs people who maintain it. Both are part of the job.
Platform
ML Systems Lab
One config file, one command, every machine on the bench. A reproducible benchmarking framework spanning laptops, Raspberry Pi and GPUs, with per-rail power, thermal and utilization telemetry in every record.
Reproduces the published roofline fits exactly; independent campaigns agree within 1.7%
DOI 10.5281/zenodo.21867055
Upstream
Fixes to the field's infrastructure
Merged fixes in NVIDIA's garak and ai-dynamo's AIPerf, a merged Arm Learning Path, and open pull requests to vLLM: telemetry, harness correctness, benchmark robustness, JIT preflight. Root-cause habits, applied upstream.
8 merged, 15 open across AIPerf, vLLM, garak, Arm Learning Paths
ai-dynamo, NVIDIA, Arm, vllm-project
Recent notes.
- 23 Sept 2026 I had five benchmark harnesses and they did not agree with each other Every paper grew its own set of scripts, and by the fifth one I could no longer tell whether a difference between two results was the hardware or the harness. Rebuilding them as one framework was the only way to find out.
- 23 Sept 2026 Your local language model is waiting on memory, and one command will prove it Text generation on a CPU is a memory problem wearing a compute problem's clothes. Here is the one-line rule that predicts it, the evidence from seven models on one laptop, and a tool that measures it on your machine.
- 29 Aug 2026 Kubernetes did not slow my benchmark down. One line of YAML took 64 percent. I moved an inference benchmark into a pod expecting to pay for the container. Across fifty runs I could not find a cost at all. What did cost me was a CPU limit set below the thread count, which is a thing you can write by accident and never notice. →