Ilya's First Model Revealed: SSI's Test-Time Training Engine

A leak on X by user 'Three Strawberries' claims that Ilya Sutskever's startup SSI is exploring a small reasoning engine based on Test-Time Training (TTT). The model uses specially curated data to learn how to learn, then updates partial weights during problem-solving, allowing even a small model to compete with larger ones. The leaker suggests the current version is ready and the next version will scale up tenfold, with the current version possibly debuting as early as August to a limited number of users. This aligns with Ilya's past statements about continual learning and AGI, his belief that superintelligence should learn after deployment, and Nvidia's recent announcement of a strategic partnership and investment in SSI, which increased SSI's compute by an order of magnitude. The TTT approach contrasts with conventional context extension, as it treats context as training data, allowing the model to change itself during inference, though it raises safety concerns, which the leaker claims Ilya has addressed. SSI, founded in 2024, has been secretive, but this leak suggests a potential breakthrough. The news also references a paper 'End-to-End Test-Time Training for Long Context' as related work.