Research library

Findings, working logs, tools and theory. Everything measured is published with the script that produced it and the hardware it ran on; anything not yet measured is filed under Theory and labelled as such.

Findings 6

Measured results, with the number and the method.

The activation subspace is not low-dimensional

A 41-token probe suggested 16-28 dimensions of 3072. At 318 tokens it is 161-183. The dimensionality tracks the sample, not the model.

2026-08-16ranksubspacenegative-result

The 17th oscillator — an antagonist that prevents synchronisation

A central repeller that pushes every ring node away while fleeing their mean phase. Mode collapse solved by making the fixed point unreachable rather than removing it.

2026-08-16oscillatorskuramotomode-collapse

Ternary weights do not survive, and per-column scaling does not rescue them

46% of the output is lost on a single matrix. The scale can carry the exponent, but ternary discards the mantissa entirely.

2026-08-16ternaryquantisationnegative-result

SwiGLU has no exploitable contextual sparsity

Half the FFN channels carry 90% of the magnitude and the active set churns 70-90% per token. The published wins for this technique are on ReLU models.

2026-08-16sparsityswiglunegative-result

Quantisation fidelity is a property of bit width, not of the model

Llama-3.2-3B and Gemma-4-12B agree to three decimal places at every bit width tested, including on a model trained quantisation-aware.

2026-08-16quantisationint8ternary

Massive activations invalidate naive rank measurements

Removing 8 channels out of 3072 moves measured effective rank from 1.1 to 15.0. Any compression analysis that skips this step will conclude the model is rank-1.

2026-08-16methodsactivationsrank

Logs 1

Working sessions: what was tried, what failed, what was decided.

A night of CPU inference work, including everything that failed

Ten hypotheses, three survivors, one benchmark claim retracted. The honest version of an optimisation session.

2026-08-16inferencecpuoptimisation

Code 1

Tools and implementations, and what each is for.

Six measurement tools that need no GPU

Each answers a question usually guessed at, reads weights directly, and runs on a CPU in seconds to minutes.

2026-08-16toolsopen-sourcereproducible

Theory 1

Arguments and models. Marked clearly as not yet measured.

The forward pass is one fixed point, and two solvers agree exactly

Jacobi and Gauss-Seidel converge to identical logits, error 0.000e+00. A structural result about transformers, with no speed claim attached.

2026-08-16equilibriumDEQarchitecture