Skip to content
MN

//Lab

Where the numbers come from.

Every claim on this site traces back to hardware sitting on a desk. This page is the machines, the on-board instrumentation they already carry, and the rules a result has to survive before it gets written down.

Machines
4
On the bench
3
Rules
6
Telemetry
13 h open dataset

One diagram explains most of it.

The roofline. Performance climbs with arithmetic intensity until the machine runs out of memory bandwidth or compute, whichever comes first. The research program is, in one sentence: real edge workloads live on the left slope.

Roofline schematic A roofline diagram. Performance rises along a bandwidth-limited diagonal, then flattens at a compute ceiling past the ridge point. Typical edge neural-network workloads are marked on the bandwidth-limited side, well to the left of the ridge. MobileNetV3-Small ResNet-18 EfficientNet ViT MEMORY BOUND COMPUTE BOUND RIDGE POINT REAL EDGE DNN INFERENCE LIVES HERE ARITHMETIC INTENSITY ( FLOP / BYTE ) ATTAINABLE PERFORMANCE
Schematic, not a data plot. The slope on the left is memory bandwidth; the flat section on the right is peak compute. Almost every neural network worth running on an edge device sits on the left of the ridge, which is why bandwidth, not FLOPs, predicts what it will do.
9
CNNs + 1 ViT characterised
13 h
Open telemetry dataset
0.98
Decode roofline R², Arm and x86
2 of 9
INT8 results that reverse sign across ISAs

The bench.

Small, deliberate, and instrumented without adding a single sensor: the power telemetry and performance counters are already on the die. A short list of machines that are actually measured beats a rack of machines that are merely owned.

A Raspberry Pi 5 single-board computer on a cutting mat, showing the Broadcom BCM2712 SoC, the LPDDR4X package, the 40-pin header and the PCIe connector

Raspberry Pi 5

Primary edge-inference bench

Active
  • › Broadcom BCM2712, 4x Arm Cortex-A76
  • › LPDDR4X, shared-bandwidth memory subsystem
  • › On-board PMIC power telemetry
  • › Hardware performance counters via perf

Every published edge-inference number on this site came off this class of machine.

Cisco UCS C220 / C240

Server-class measurement

Planned
  • › Rack servers, NUMA topology
  • › CPU inference and memory-hierarchy work
  • › Power measurement at the chassis

Framed to the hardware vintage: NUMA, memory wall, CPU-inference energy. Deliberately not a modern-LLM story.

i7-12700H laptop

Cross-architecture check, and the hybrid-core work

Active
  • › 6 performance cores, 8 efficiency cores, DDR5
  • › RTX 3050 Laptop, treated as a separate device by the harness
  • › Intel RAPL energy counters
  • › Roughly four times the memory bandwidth of the Pi

This machine is why the results are not a Raspberry Pi story. The memory-bandwidth law was re-measured here on a completely different architecture and held with the same goodness of fit, and the mixed performance and efficiency cores are the subject of their own paper. The GPU in it used to be listed separately, which flattered the bench: it is one laptop.

iMac G3, 1998

Because it is there

Active
  • › Bondi blue
  • › Boots

Not part of any experiment. It sits on the bench and it works, which is the point.

Rules a result has to survive.

Written down because they cost something. Each of these has, at least once, meant deleting a result that would have made a better headline.

  1. 01

    Measure the machine, not the toolchain

    A benchmark that does not control for its own toolchain is measuring the wrong thing. This rule is here because I broke it: I once reported a fortyfold latency swing and attributed it to the export format, and one arm of that experiment had quietly loaded a model quantized a different way instead of building the one it claimed to build. The swing was real and the cause was wrong. Two later papers exist to correct it.

  2. 02

    Record the state of the machine, not just the number

    Every run writes down the clock speed, the temperature, the throttle bits and the power draw alongside the result. That caught five measurements taken while something else was running on the board, with a run-to-run spread of 30 to 103 percent against a campaign norm under 2.5 percent. Without the record they would have gone quietly into an average.

  3. 03

    Publish the negative result

    When a headline number failed to reproduce, the paper was rewritten to whatever the data supported. Two earlier drafts were found to present simulation as measurement and were discarded rather than published.

  4. 04

    Ship the harness

    Every published result has a public repository with the code and data that produced it, archived on Zenodo with a citable DOI. If a reviewer cannot re-run it, it is an anecdote.

  5. 05

    Instrument power, do not model it

    Energy claims come from on-board power telemetry sampled during the run, not from a thermal design power figure multiplied by a duration.

  6. 06

    State the regime

    Most control policies help in some operating regimes and hurt in others. Reporting the boundary is more useful than reporting the win.