AMD acquires Taalas: startup burns AI models into chips for fast inference

AMD is acquiring the Canadian AI startup Taalas, which builds specialized inference chips. Founded in 2023 in Toronto, Taalas emerged from stealth in February. The startup embeds the structure and learned parameters of an AI model directly into the chip, making inference extremely fast, but the chip can only run a single model. A demo chip achieved more than 16,000 tokens per second per user with Llama3.1-8B, several times faster than current competing chips. Google is reportedly also working on such a chip for its Gemini models. AMD wants to integrate the technology into its accelerator roadmap and offer it together with Instinct GPUs as a system solution. According to Vamsi Boppana, Senior Vice President of AMD's AI division, the acquisition strengthens the AI portfolio. Taalas co-founder Ljubisa Bajic said AMD offers the necessary scale and reach. The acquisition is subject to customary approvals.