Attention on the Null Cone

Trained transformer attention exhibits power-law decay in attention weight as a function of query–key separation, with a per-head exponent whose median in deep GPT-2 layers matches the Sachdev–Ye–Kitaev (SYK) conformal value Δ = 1/4. Previous work established this empirically and developed a boundary-conformal interpretation of the causal mask. Here we identify the geometric home of that conformal structure. The conformal group is not merely a symmetry that happens to include scale transformations: by a theorem of Lorentzian geometry, it is precisely the group of transformations preserving the null cone — the causal structure of light. In the embedding-space formalism, conformal-field-theory boundary points are null rays, and the conformal two-point function is the inner product of null-ray representatives. We show that the one-dimensional case relevant to token sequences admits an explicit Poincar'e section 𝑃 (𝑥) = ( 1+𝑥2 1−𝑥2 2 , 2 , 𝑥) on the null cone of ℝ2,1, under which the attention two-point function 𝐴(𝑖, 𝑗) ∼ |𝑖 − 𝑗|−2Δ is exactly the CFT1 two-point function. This predicts that the pre-softmax query–key score should be log-linear in token distance, 𝑞𝑖 ⋅ 𝑘𝑗/√𝑑𝑘 ≈ −2Δ log |𝑖 − 𝑗| + const. We test the prediction directly in GPT-2 (exp-056): the raw query–key score profile is log- linear in distance for every conformal head, and the slope Δscore = −𝛼/2 rank-correlates with the independently measured post-softmax exponent at Spearman 𝜌 = +0.976 (𝑝 = 2 × 10−29). Two honest qualifications follow from the data: the softmax normalization renormalizes the coefficient (the raw slope corresponds to ≈ −2.8Δ, not −2Δ), and log-linearity itself is a near- universal property of the positional geometry rather than a feature selective to conformal heads — the conformal dimension is the selective quantity. We also derive, by the method of images, a boundary-corrected power-law form for the causal-mask correction, and test it directly (exp- 057): the derived form fits the position-dependence of attention, with a boundary coefficient that is positive in 95% of conformal heads—the direction of the attention sink. An adversarial model comparison against matched-complexity alternatives (exp-060) shows that an exponential boundary layer outperforms the image form; the boundary correction is real and large, but its specific BCFT parametrization is phenomenological. Finally, a layer-resolved analysis (exp- 055) shows the per-head exponent flows from̄ Δ = 0.70 in early layers tō Δ = 0.250 in deep layers, with attention entropy increasing monotonically along the flow (𝜌(Δ, entropy) = −0.898, 𝑝 = 10−16). We read layer depth as renormalization-group flow toward the SYK conformal vacuum, dual to the eternal AdS2 black hole. All code and per-head data are in the public repository at Capacity-For-Evil/ariel.