Hugging Face

Latest AI news, models and releases from Hugging Face. ['Diffusers', 'GLM 5.2', 'LeRobot v0.6.0', 'Nunchaku Lite', 'qwen3-32b', 'Qwen/Qwen3-4B', 'Qwen/Qwen3.5-122B-A10B', 'Qwen/Qwen3.5-4B', 'SmolLM', 'SmolLM2-1.7B', 'SmolLM3', 'SmolLM3 3B', 'SmolLM3-3B', 'SmolVLM-256M', 'TinyLlama', 'tokenizers', 'Transformers v4', 'Transformers v5', 'TRL', 'Xenova/distilbert-base-uncased-finetuned-sst-2-english', 'Xenova/whisper-tiny.en']

Models 🇷🇺

Building Your Own Warhammer 40K GPT-like LLM from Scratch. Part 4: Supervised Fine-Tuning on Question-Answer

The fourth part of a series on creating a decoder-only LLM from scratch, focusing on supervised fine-tuning (SFT) on a question-answer dataset to enable dialogue. The author extends the tokenizer with special role tokens, builds a QA dataset, and fine-tunes the model, observing that while loss decreases, generation quality may degrade due to overfitting. The checkpoint is selected based on generation quality rather than minimal loss.

Hugging FaceHugging Face
Habr — хаб ИИ24.07 · 10:02
Fresh news