Meta

Latest AI news, models and releases from Meta. ['Astryx', 'Business Agent', 'BYT5', 'CodeLlama', 'expressive voice AI', 'FAIRChem v2 UMA', 'Gemma 4', 'Gemma4:e2b', 'HuBERT', 'llama', 'Llama', 'LLaMA', 'Llama-2 70B', 'LLaMA2-chat-70B', 'Llama 3', 'Llama 3.1', 'Llama 3.1-405B', 'Llama-3.1-405B', 'LLaMA-3.1 405B', 'Llama 3.1-70B', 'Llama 3.1 8B', 'Llama 3.1-8B', 'Llama-3.1-8B', 'Llama 3.2', 'LLaMA 3.2', 'Llama-3.2-1B-Instruct', 'Llama 3.3', 'llama-3.3-70b', 'Llama 3 8B', 'Llama3-8B-Instruct', 'Llama 4', 'Llama 4 Maverick', 'LLaMA 65B', 'Llama 70B', 'M2M100', 'M**a AI', 'mBART', 'Meta AI', 'Meta AI model (model name not specified)', 'Meta Muse Spark']

Official site ↗

Research 🇷🇺

Writing a Decoder-Only Transformer for LLM from Scratch in Python

The author of a series of articles describes in detail the implementation of a transformer block for a small decoder-only LLM using the PyTorch framework. Components covered include: masked Multi-Head Attention, Feed Forward Network with GELU, normalization layer, and residual connections. The article explains the difference from the original architecture: pre-normalization (Pre-LN) is used instead of post-normalization for training stability.

Google/DeepMind

OpenAI

Anthropic criticized for refusing open models — only major AI lab without them

Anthropic is the only major AI lab that does not release open models, drawing criticism. Other leading labs like Google DeepMind, Meta, and Mistral support open models.

Anthropic

OpenAI