Transformers
& Large Language Models

RNNs and LSTMs process sequences one token at a time. Each step depends on the previous hidden state, making parallelisation impossible. This created a hard ceiling:

• Training speed — cannot leverage GPU parallelism; one step must complete before the next begins.
• Long-range dependencies — even LSTMs struggle to connect information separated by hundreds of tokens; gradient signal degrades over long paths.
• Scale barrier — training on internet-scale datasets (trillions of tokens) was simply not feasible with sequential architectures.

The Transformer, introduced in "Attention Is All You Need" (Vaswani et al., 2017), replaces recurrence entirely with attention. Every token attends to every other token in a single parallel operation. The consequence was transformative: training on trillions of tokens became feasible, enabling GPT, BERT, and every major LLM that followed.

Attention was first introduced in 2015 to improve neural machine translation by creating shortcuts between the encoder and the decoder, solving issues with long input sequences.

Attention dynamically computes relevance scores between all words in a sentence, allowing the model to focus on the most important words within the current context and combine their meanings. The result is context-aware representations.

Classic example — the word "bank":

• "I deposited money at the bank" — attention weights concentrate on "money", "deposited".
• "I sat by the river bank" — attention weights concentrate on "river", "sat".

Same word, different representation — this is what static embeddings (Word2Vec, GloVe) cannot do.

Self-attention transforms each token into three vectors by multiplying its embedding by three learned weight matrices:

• Query (Q) — "What am I looking for?"
• Key (K) — "What information do I offer?"
• Value (V) — "What content do I actually provide?"

Step-by-step breakdown:

1. Dot products: Compute $QK^T$ — the raw relevance score between every query and every key.
2. Scale: Divide by $\sqrt{d_k}$ (key dimension) to prevent vanishing gradients from large dot products.
3. Softmax: Convert scores to probabilities — the attention weights sum to 1.
4. Weighted sum: Multiply weights by $V$ — the output is a weighted combination of all value vectors.

Before self-attention, each token passes through:

1. Token embedding — maps the token ID to a dense vector (e.g. 512-dimensional).
2. Positional encoding — adds a position-dependent signal so the model knows token order (since there is no recurrence). Uses sine/cosine functions of varying frequencies: $PE_{(pos, 2i)} = \sin\!\left(\frac{pos}{10000^{2i/d_{model}}}\right)$

Single-head attention acts like one giant spotlight that can only point in one direction, often focusing on the most obvious connection while missing others. Multi-head attention runs $h$ attention heads in parallel, each with its own $W^Q_i, W^K_i, W^V_i$ projection matrices.

Different heads specialise in different linguistic relationships simultaneously:

The outputs of all heads are concatenated and projected through $W^O$ to produce the final multi-head attention output. The original Transformer (Vaswani 2017) used $h = 8$ heads, $d_{model} = 512$, so each head operates on $d_k = d_v = 64$ dimensions.

Each encoder layer consists of two sub-layers with residual connections and layer normalisation (Add & Norm) around each:

1. Multi-Head Self-Attention — every token attends to all other tokens.
2. Feed-Forward Network (FFN) — a position-wise two-layer MLP applied identically to each token.

The FFN has a larger inner dimension ($d_{ff} = 2048$ in the original) than $d_{model} = 512$. It expands then contracts — like a "thinking" step after attention. The same FFN weights are applied to every position independently.

Each decoder layer has three sub-layers:

1. Masked Multi-Head Self-Attention — tokens can only attend to previous positions (causal masking prevents looking ahead during training).
2. Multi-Head Cross-Attention — queries come from the decoder, keys and values come from the encoder output. This is how the decoder "reads" the encoded source sequence.
3. Feed-Forward Network — same as in the encoder.

After the final decoder layer: Linear projection to vocabulary size → Softmax → probability distribution over the next token.

The original Transformer architecture was designed for sequence-to-sequence tasks (e.g. translation). The decoder generates output autoregressively — one token at a time:

1. Start with the <BOS> (beginning-of-sequence) token.
2. Run it through the decoder; output a probability distribution over the vocabulary.
3. Sample or select the highest-probability token (decoding strategy).
4. Append the new token to the sequence and repeat from step 2.
5. Stop when the <EOS> token is generated or max length is reached.

During training, the decoder uses teacher forcing: all ground-truth tokens are fed in parallel (with causal masking), making training efficient. During inference, the model generates tokens one-by-one using its own previous outputs.

LLMs are Transformer models trained at a scale previously unimaginable: billions of parameters, trillions of tokens, thousands of GPUs over months. The architecture is the same Transformer block — stacked, wider, and deeper.

Beyond predictable performance improvements, large models exhibit emergent abilities — capabilities that appear suddenly at scale and were not present in smaller models:

• Multi-step reasoning — solving multi-step math problems without explicit reasoning training.
• In-context learning — learning from a few examples in the prompt without updating weights.
• Chain-of-thought — generating coherent reasoning chains to improve accuracy.
• Instruction following — understanding and executing complex natural language instructions.
• Code generation — writing functional code across many programming languages.

Every major production LLM follows the same three-stage training pipeline. Each stage converts the raw architecture into a more useful, aligned assistant.

Objective: Next-token prediction (autoregressive) on massive, diverse text data.

• Data: web crawls (Common Crawl), books, Wikipedia, code, scientific papers.
• Outcome: A model that knows language, facts, reasoning patterns — but has no instruction-following behaviour.
• Cost: Millions of dollars in compute; done once by the lab, not practitioners.

Objective: Teach the model to follow instructions using high-quality human-written examples.

• Data: Curated prompt-response pairs, written or verified by humans.
• Outcome: Model learns the format and style of helpful, coherent responses.
• Scale: Much smaller dataset than pre-training (thousands to millions of examples).

Objective: Align model outputs with human values — helpfulness, harmlessness, honesty.

• RLHF (Reinforcement Learning from Human Feedback): Human raters rank multiple model outputs; a reward model is trained on those rankings; the LLM is optimised against that reward model using RL (PPO).
• DPO (Direct Preference Optimisation): A simpler alternative that skips the explicit reward model.
• Outcome: The model refuses harmful requests, stays on-task, communicates clearly.

The original encoder-decoder Transformer has spawned a family of variants, each designed for a specific use case by keeping or removing parts of the architecture.

Zero training data. Zero infrastructure changes. No compute cost. Just better instructions. Prompt engineering is the first tool every practitioner should exhaust before reaching for more expensive approaches.

Sets the model's role, persona, constraints, and style before any conversation begins. The same base model becomes a customer service agent, code reviewer, or research assistant depending on the system prompt.

Give the model a task description with no examples. Works well for common tasks the model encountered during training.

Show 2–3 concrete examples of input → desired output. The model infers the pattern and applies it. Works through in-context learning — an emergent capability. Weights are never updated.

For reasoning tasks (math, logic, multi-step decisions), ask the model to "think step by step" before answering. Generates intermediate reasoning steps that guide the final output. Small prompt change, significant quality gain.

RAG solves two critical limitations of bare LLMs:

• Stale knowledge — LLMs have a training cutoff; they cannot know about recent events.
• Hallucination — LLMs generate plausible but sometimes incorrect outputs with no external grounding.

1. Index — chunk your documents, embed them with a text embedding model, store in a vector database.
2. Retrieve — when the user sends a query, embed the query and search for the most similar document chunks (cosine similarity).
3. Augment — prepend the retrieved chunks to the LLM's prompt as context.
4. Generate — the LLM answers the query grounded in the retrieved context, dramatically reducing hallucination.

You can give any LLM access to private, up-to-date, domain-specific knowledge without modifying its weights. The retrieval backbone uses the word embeddings you learned in Module 05. RAG is how most enterprise LLM applications are built.

Fine-tune when a specific domain expertise, tone, style, or structured behaviour cannot be reliably produced by prompting and RAG alone.

Often done sequentially: domain adaptation first, then task fine-tuning on top. Choose based on the failure mode — wrong vocabulary or wrong behaviour.

Full fine-tuning updates all billions of parameters — prohibitively expensive for most practitioners. PEFT techniques update only a tiny fraction of parameters while keeping the original weights frozen, achieving high-performance adaptation at a fraction of the cost.

Model rankings change monthly. This decision framework does not.