Quantisation fidelity is a property of bit width, not of the model

Two model families, four bit widths, measured by reconstructing real weight matrices and comparing the product W·x against the unquantised original. No forward pass is involved, so no engine can contaminate the result.

The two columns track to three decimal places at every width. Weight distributions are approximately Gaussian across model families and scales, so per-column symmetric quantisation lands in the same place regardless of size or training regime.

This includes a QAT model. The Gemma checkpoint was trained quantisation-aware at q4_0. At four bits it reaches 0.981 — the width it was trained for. Below that it is extrapolating past its own training and gains nothing over a model that never saw quantisation.

Consequence

A measurement on a small model generalises to a large one, which makes low-bit research much cheaper than it is usually treated as being. It also means "quantise a bigger model and it will hold up better" is not supported.

Reproduce with bitwidth_sweep.py. Runs on a CPU in a few minutes.