Large Language Models

Modern text generation is dominated by the Decoder-Only Transformer, trained primarily on next-token prediction.

The Lifecycle: Pretraining, Post-training, and Inference

A modern LLM undergoes distinct lifecycle stages that dictate what interventions are possible:

• Pretraining: Ingesting trillions of tokens to learn grammar, facts, and reasoning circuits. Produces a base model that only knows how to complete documents, not answer questions.
• Post-training (Alignment): Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF) teach the model to act as a helpful assistant, respond to instructions, and follow safety constraints.
• Inference: The deployment phase where the frozen weights process user prompts and sample outputs.

Architecture: The Transformer Backbone

Avoid anthropomorphizing "attention." Self-attention is a learned routing mechanism. It allows a token representation at layer N to mix in information from previous tokens before passing to layer N+1. It does not "understand" text; it performs a mathematically optimized information retrieval over the context window.

During inference, models utilize a KV Cache (Key-Value cache). Instead of recalculating attention for the whole prompt on every single generated word, the model caches the intermediate states of previous tokens, drastically reducing compute at the cost of high memory usage.

Sampling: Temperature & Top-P

The neural network outputs a vector of raw scores (logits) for every possible next token. We convert these to probabilities using a softmax function, modified by Temperature and Top-P (Nucleus Sampling).

• Temperature: Divides logits before softmax. `T=1.0` is natural. `T < 1` sharpens the distribution. `T > 1` flattens it.
• Top-P: After probabilities are calculated, the smallest set of likely tokens whose cumulative probability reaches P is retained and renormalized. A lower value truncates more of the tail; it can reduce surprising continuations, but does not guarantee coherence.