From LLM to Agent: Understanding the Layers Between Model and Application

The article describes the complete path from an LLM to an agent. At the core, the model predicts the next token in an autoregressive loop, using a tokenizer to convert text to token IDs and a detokenizer to convert back. KV-cache accelerates generation by storing previous key-value pairs. The input text expected by the model is not raw JSON but a formatted prompt according to the model's chat template, which is a Jinja template that converts a list of messages into a string with special tokens (e.g., ChatML, Llama 3). For reasoning models, the prompt may include a <think> block; if the model spends too many tokens on reasoning, it may run out of generation budget before producing a final answer. The article also explains that billing and context length are based on the tokenized prompt after the chat template, not the API JSON. Model storage on disk includes safetensors weights, config.json, tokenizer files, and the chat_template in tokenizer_config.json. Runtimes like llama.cpp load a GGUF file that bundles these components, providing a high-level interface from text to text.