The Compiler That Hallucinates: Compiling Programs Straight into Neural Network Weights

A compiler traditionally translates high-level code into primitive machine instructions tailored to specific hardware. In contrast, a new paradigm treats neural networks as the target architecture: programs written in a specialized language like RASP are compiled directly into weights, deterministically and without training. RASP, introduced in 2021 by Gail Weiss, Yoav Goldberg, and Eran Yahav, provides two main constructs—select and aggregate—mirroring the attention and feed-forward mechanisms of transformers. In 2023, DeepMind implemented the Tracr compiler, which takes RASP programs and outputs transformer weights, using the residual stream as memory and allocating disjoint chunks for variables. These compiled models are fully interpretable: every variable's location and each layer's function are known. The primary motivation is to create ground-truth models for interpretability research, serving as calibration standards for analysis tools. A notable finding is that Tracr's straightforward variable allocation is wasteful compared to real networks, which densely overlap information, suggesting that trained networks develop more efficient but messier representations.