The forward pass is one fixed point, and two solvers agree exactly

A transformer forward pass is normally described as 28 sequenced layers. It is equally a single fixed point: z = F(z; x), where F is the whole network's coupling rule and the answer is the state that satisfies it.

This is not a reinterpretation offered without evidence. Two solvers that share no structure were run on the same input:

• Gauss-Seidel — visits blocks in topological order. One sweep suffices,

because a DAG in dependency order is exactly a forward pass.

• Jacobi — updates every block simultaneously from the previous state.

Nothing is sequenced.

They converge to identical logits, relative error 0.000e+00, at sweep 19 for a two-layer system — exactly the critical path depth. Jacobi needs depth sweeps because information must physically propagate through every stage.

What follows, and what does not

Does not follow: any speedup. Gauss-Seidel in topological order is the ordinary forward pass. The equilibrium framing is a true statement about the arithmetic, not a cheaper way to perform it. Collapsing the relaxation to its algebraic limit is what the shipped code does, and that collapse is exact.

Does follow: a training method. For a system defined by a fixed point, gradients come from implicit differentiation at the solution — you never differentiate through the iterations, so activations are never stored. Memory becomes constant in depth: 12.6 MB against 1.29 GB for standard backprop at batch 4, and a 48-layer model costs the same as a 28-layer one.

The backward pass is itself a settle, and the vector-Jacobian product streams the weights in the same order the forward does — no transpose, no second copy.

Filed under theory: the equilibrium result is measured, the training method that follows from it is designed but not yet implemented.