Reference

Glossary

Every term the book uses, in plain words. Each entry names the chapter that teaches it in depth.

Activation function: A simple non-linear function applied to a neuron's weighted sum, such as ReLU or the sigmoid. Without it, a stack of layers collapses into one straight-line function. See chapter 12.
Adam: An optimiser that keeps a running average of each weight's gradient (momentum) and of its squared gradient, and scales each step by both. The default choice for training transformers. See chapter 13.
adapter: A small set of trained weights added to a frozen model, such as LoRA's A and B matrices; one base model can carry a different adapter per task. See chapter 40.
Agent: A program in which a language model chooses actions (usually tool calls) in a loop, sees the results, and decides what to do next until a goal or a stopping rule is reached. See chapter 29 and Lab 5.
Agent loop: The cycle at the centre of an agent: send the conversation and the tool descriptions to the model; if it asks for tools, run them and append the results; call the model again; stop when it answers without asking for a tool or a step limit is reached. See chapter 29 and Lab 5.
Allow-list: A fixed list of what an agent may use, such as the tools it may call. Anything not on it is never offered to the model. See chapter 35 and chapter 28.
Approximate nearest neighbour (ANN): A search that examines only part of the stored vectors and usually, not always, finds the true nearest ones. IVF and HNSW are the common designs. See chapter 25.
As-of date: The date a figure or fact was true. A number with no date can be years out of date, so a careful system stores the date with each figure and shows it beside the number. See chapter 27.
Attention: The transformer's way of letting each token gather information from other tokens, weighting them by how well its query matches their keys. See chapter 18.
Autoregressive: Producing a sequence one piece at a time, each piece conditioned on the pieces already produced. Language models generate text this way. See chapter 16.
AWQ: Activation-aware weight quantisation: protects the few weights that matter most (found from activation sizes) by scaling them before rounding. See chapter 39.
Backpropagation: The method that computes every weight's gradient in a network by applying the chain rule backwards from the loss, layer by layer. See chapter 12.
Batch: A group of training examples processed together in one step. The gradient is averaged over the batch. See chapter 6.
Batch normalisation: A layer that rescales each feature to mean 0 and variance 1 across the examples in a batch, then applies a learned scale and shift. See chapter 13.
Bayes' rule: A formula that turns P(B given A) into P(A given B) using the base rate P(A). See chapter 7.
bf16: "Brain float 16": a 16-bit number format with the same exponent range as fp32 but fewer digits of precision. Common for training. See chapter 39.
bge-small: BAAI/bge-small-en-v1.5, a small open embedding model that turns text into 384 numbers and runs on an ordinary laptop. See chapter 23.
Bias (of a neuron): The number a neuron adds to its weighted sum before the activation, letting it shift its output up or down. Not the same as unfairness or statistical bias.
Bias probe: A test that changes how an answer is presented (order, length) but not what it says, and checks that a judge's scores stay the same. See chapter 44.
Bi-encoder: A model that encodes a query and a passage separately into vectors, so passage vectors can be computed ahead of time. Embedding models are bi-encoders. See chapter 25.
Bigram model: The simplest language model: it predicts the next character or word from the one before it only, using counts. See chapter 20.
BM25: The standard keyword relevance score: rare words count more, repeated words count with diminishing returns, and long documents are discounted. See chapter 25 and Lab 4.
BPE (byte-pair encoding): A way to build a tokeniser's vocabulary: start from single characters or bytes and repeatedly merge the most frequent neighbouring pair into a new token. See chapter 16.
catastrophic forgetting: A model losing an old skill while it is trained on a new one. See chapter 40.
Chain rule: The rule for the derivative of a function inside a function: multiply the derivatives of each stage. The heart of backpropagation. See chapter 5.
Chunk: A piece of a document small enough to embed and retrieve on its own. A chunk whose text never names its subject is hard to find, so it can be embedded with a label in front (see labelled chunk). See chapter 24 and Lab 3.
circuit breaker: A switch that stops calling a failing dependency for a while after repeated errors, then tries again. See chapter 42.
Citation: A pointer from a claim to the document section it came from. A careful system checks that any quoted text really appears in the cited place, and labels the passage by the heading printed before it or by its page number. See chapter 24.
Cohen's kappa: A measure of how much two raters agree beyond what chance would give: 1 is perfect agreement, 0 is chance level. An LLM judge's agreement with human labels is measured with it. See chapter 44 and Lab 8.
Compaction: Replacing the oldest part of a long conversation with a model-written summary, so the window keeps decisions and open questions without every earlier token. See chapter 22.
Conditional probability: The probability of one thing given that another is known to be true, written P(A | B). See chapter 7.
Confidence interval: A range that, by a stated method, captures the true value of a quantity most of the time (for example 95% of the time). Small samples give wide intervals. See chapter 36.
Context engineering: Choosing what goes into the context window on each call: which history, documents and tool results, in what order and how much. See chapter 22.
Context variable: A Python value (contextvars.ContextVar) that each task sees its own copy of, so concurrent requests do not mix. Tracing and cost code use them to know which request a call belongs to. See chapter 45.
Context window: The most tokens a model can read at once, prompt and answer together. Anything outside it is invisible to the model. See chapter 16.
continuous batching: Adding new requests to a batch that is already running, instead of waiting for the whole batch to finish. See chapter 38.
Contrastive learning: Training that pulls the vectors of matching pairs together and pushes non-matching pairs apart. Most embedding models are trained this way. See chapter 23.
Cosine distance: One minus the cosine similarity. Vector databases such as pgvector return it as the distance for their <=> operator, so the similarity is 1 minus that distance. See chapter 23.
Cosine similarity: The cosine of the angle between two vectors: the dot product divided by both lengths. 1 means same direction, 0 means unrelated, -1 means opposite. See chapter 23.
Cost rollup: The cost and tokens of every model call under a span, added up and written on the spans above it, so one span shows what everything inside it cost. See chapter 45 and Lab 9.
Cross-encoder: A model that reads a query and one passage together and outputs a relevance score. More accurate than a bi-encoder and too slow for whole collections, so it reranks a short list. See chapter 25.
Cross-entropy: A loss that measures how surprised a model's predicted probabilities are by the true answer: minus the log of the probability it gave the right answer. See chapter 9.
Derivative: The slope of a function at one point: how fast the output changes per unit change of the input. See chapter 5.
Dimension: The number of entries in a vector. An embedding with 384 dimensions is a list of 384 numbers. See chapter 4.
Disclosure: Fixed wording an answer must carry in some situations, such as a notice that it is not professional advice. Code adds it, so the model cannot forget or reword it. See chapter 34.
Distillation: Training a small "student" model to imitate a larger "teacher" model's output probabilities rather than only the right answers. See chapter 40.
Dot product: Multiply two vectors entry by entry and add the results. It is large when the vectors point the same way. See chapter 4.
DPO: Direct preference optimisation: tunes a model on pairs of preferred and rejected answers with a simple loss, without training a separate reward model. See chapter 19.
Dropout: During training, randomly switch off a fraction of neurons at each step, so the network cannot depend on any one of them. See chapter 13.
Probability distribution: A list of possible outcomes with a probability for each, adding up to 1. See chapter 7.
Effort (thinking effort): A setting, usually low, medium or high, that tells a reasoning model how much to think before it answers. Higher effort helps on hard problems and costs more, because the thinking is written as reasoning tokens. See chapter 16 and Lab 1.
Elicitation: An MCP feature that lets a server ask the user for input, through a form or by sending them to a URL. A form must never ask for passwords, keys or payment details. See chapter 30.
Embedding: A list of numbers (a vector) that stands for a piece of text, so that texts with similar meaning get vectors pointing in similar directions. See chapter 23.
Embedding layer: The first layer of a language model: a table with one learned vector per token id. See chapter 17.
Entropy: The average surprise of a distribution, in bits or nats. High when outcomes are evenly spread, zero when one outcome is certain. See chapter 9.
Epoch: One full pass over the training data. See chapter 6.
Eval: A repeatable test of an AI system on a fixed set of cases, with a scoring rule. See chapter 44.
Exponential: A function like 2^x or e^x where the input is the power. It grows by the same factor for each step of x. See chapter 3.
exponential backoff: Waiting longer after each failed attempt (1, 2, 4, 8 seconds), usually with random jitter, before retrying. See chapter 42.
Factuality: Whether a statement is true about the world. Compare faithfulness. See chapter 36.
Fail closed: Treating a check that could not run as a failed check, so an error never lets output through unchecked. See chapter 33.
Faithfulness: Whether a statement is supported by the sources the model was given. A judge usually scores it as a criterion of its own. See chapter 36.
Fan-out: Starting several branches of work at once from one point, such as running several subagents in parallel and merging their reports. See chapter 32.
fastembed: A small Python library that runs embedding models on the local machine, with no API key. See chapter 23.
Few-shot prompting: Putting a few worked input-and-output examples in a prompt so the model copies the format and judgement. Models also copy the examples' surface features, so vary them. See chapter 21.
Fine-tuning: Training an already-trained model a little more, on a smaller dataset, to change its behaviour. See chapter 40.
fp16: 16-bit floating point: 1 sign bit, 5 exponent bits, 10 fraction bits; largest value 65,504. See chapter 39.
fp32 and fp16: Floating-point number formats with 32 and 16 bits. fp32 is the usual default; fp16 halves the memory at the cost of range and precision. See chapter 39.
Function: A rule that turns each input into exactly one output, like a machine. See chapter 2.
gateway (LLM): A service or library that puts many models and providers behind one API, with routing and fallbacks. OpenRouter is one. See chapter 38.
Genkit: A framework for calling models, defining tools and getting structured output, with plugins per provider. See chapter 38.
GGUF: The file format llama.cpp uses for quantised models, holding weights in several low-bit schemes plus the tokenizer and settings. See chapter 39.
GPTQ: A post-training quantisation method that rounds weights one column at a time and adjusts the rest to cancel the error, using a little calibration data. See chapter 39.
Gradient: The list of all partial derivatives of a function. It points uphill, in the direction of fastest increase. See chapter 5.
Gradient descent: Training by repeatedly moving the weights a small step against the gradient of the loss. See chapter 6.
group-wise quantisation: Giving each small group of weights its own scale, so an outlier spoils only its group. See chapter 39.
Guardrail: A check or limit around a model that keeps the system safe even when the model misbehaves. See chapter 34.
Hallucination: Fluent model output that is not supported by the sources or not true. See chapter 36.
Harness: Everything around a model that makes it a working system: the loop, the context, tools and permissions, output checks, budgets, traces and evals. See chapter 33.
Hindsight: An open-source long-term memory server for agents: a system writes facts to it and recalls them later. See chapter 37.
Hit rate (hit@k): The share of questions with at least one relevant result in the top k, reported as hit@3 or hit@5 for k of 3 or 5. See chapter 26 and Lab 4.
HNSW: Hierarchical navigable small world: a graph index that finds nearest vectors quickly without comparing against every vector, by walking from neighbour to neighbour. pgvector can use it for its index. See chapter 25.
Hybrid search: Combining keyword search and vector search, usually by merging their ranked lists with reciprocal rank fusion. See chapter 25.
Hyperparameter: A setting chosen by a person rather than learned, such as the learning rate or the number of layers. See chapter 11.
IBAN: International bank account number. A scrubber can find one by its shape (two letters, two check digits, then 11 to 30 letters or digits) and mask it as <IBAN_A>. See chapter 34.
indirect prompt injection: Instructions planted in content a model reads (a web page, document or tool result) rather than typed by the user. See chapter 35 and Lab 7.
Inference: Using a trained model to produce outputs, as opposed to training it.
int4: 4-bit integers, 16 levels, used with a shared scale to store weights in a quarter of 16-bit memory. See chapter 39.
int8 and int4: Integer formats with 8 and 4 bits: 256 and 16 possible values. Quantised models store weights in them. See chapter 39.
Intent: What a user's message is asking for, such as a refund or the opening hours. A router can label each message with an intent to pick the prompt, tool or model that handles it. See chapter 22.
IVF (inverted file index): An approximate vector index that clusters the vectors into lists and searches only the lists whose centres are nearest the query (the probes). See chapter 25.
Jailbreak: A prompt crafted to make a model ignore its rules. See chapter 35.
Jina: A hosted embedding and reranking service. See chapter 23.
JSON Schema: A standard way to describe the shape of JSON data: its fields, types and which are required. Tools and structured outputs are described to the model with it. See chapter 28.
JSON-RPC: A simple standard for calling procedures with JSON messages: a request has an id and a method, a result or error answers it with the same id, and a notification has no id and gets no reply. MCP messages are JSON-RPC 2.0. See chapter 30.
Judge (LLM-as-judge): A separate model that scores another model's answers against a rubric. See chapter 44 and Lab 8.
KL divergence: Kullback-Leibler divergence: how many extra bits (or nats) you pay, on average, for using distribution Q when the truth is P. Zero only when they match. See chapter 9.
Knowledge base: The documents and stored facts a system reads to answer questions. See chapter 27.
Labelled chunk: A chunk embedded with a label in front, such as its document title and heading path, so a chunk that never names its subject can still be found. A reranker can read the label too. See chapter 24.
LangChain: A Python library of building blocks for model applications: model wrappers, tools and agents. See chapter 38.
Langfuse: An open-source tracing service for model applications. A program sends it each request as a trace of spans with cost and latency, and can link each answer to its trace. See chapter 45.
LangGraph: A library for building an agent or workflow as a graph: nodes (the steps) read and update a shared state, and edges say which node runs next. See chapter 38.
Layer normalisation: Rescales the numbers inside one token's vector to mean 0 and variance 1, then applies a learned scale and shift. Used in every transformer block. See chapter 13.
Learning rate: The size of each gradient descent step. Too small is slow; too large overshoots and can diverge. See chapter 6.
least privilege: Giving each component only the access its task needs, so a fooled model can do little harm. See chapter 35.
lethal trifecta: Simon Willison's name for an agent that has private data, reads untrusted content and can send data out: together they allow data theft by injection. See chapter 35.
Linear regression: Fitting a straight line (or flat plane) to data by choosing the weights that make the squared error smallest. See chapter 10.
LLM (large language model): A neural network trained on a large amount of text to predict the next token. See chapter 16.
Logarithm: The inverse of an exponential: log₂(8) = 3 because 2³ = 8. Logs turn products into sums. See chapter 3.
Logistic regression: A linear model whose output goes through the sigmoid, giving a probability for a yes-or-no answer. See chapter 10.
Logit: A raw score a model gives each option before softmax turns the scores into probabilities. See chapter 9.
LoRA: Low-rank adaptation: fine-tune a model by learning two thin matrices B and A whose product is added to a frozen weight matrix. See chapter 40.
Loss: A single number that says how wrong a model is on some data. Training makes it smaller. See chapter 10.
LSTM: Long short-term memory: a recurrent network with gates that decide what to keep, forget and output, so information can survive many steps. See chapter 14.
Masking: Replacing personal data with placeholders such as <PERSON_A> before text leaves your system, and putting it back afterwards. See chapter 34 and Lab 7.
Matrix: A rectangular grid of numbers. A layer of a neural network is mostly a matrix. See chapter 4.
Matrix multiplication: Combining two matrices by taking the dot product of each row of the first with each column of the second. See chapter 4.
MCP: Model Context Protocol: a standard way for an AI application to reach tools and data on a server. See chapter 30 and Lab 6.
Mean: The average: add the numbers and divide by how many there are. See chapter 8.
Memory bank: A named store in a memory server such as Hindsight, for example one bank per user. See chapter 37.
memory poisoning: Planting an instruction or false fact in a system's long-term memory so it is replayed into later prompts. See chapter 37.
MLP: Multi-layer perceptron: layers of neurons where each layer is a matrix multiply plus an activation. Each transformer block contains one. See chapter 12.
model card: A short document describing a model's intended use, evaluation and limits, proposed in "Model Cards for Model Reporting" (2018). See chapter 43.
Model routing: Sending each request to a model tier chosen for it, so easy requests use cheaper, faster models. A router, which can be a small model or plain rules, picks a tier such as simple, standard or complex. See chapter 22.
Momentum: An optimiser trick: keep a running average of past gradients and step along it, which smooths zig-zags. See chapter 13.
MRR (mean reciprocal rank): The average over questions of 1 divided by the rank of the first relevant result, counting 0 when there is none. See chapter 26 and Lab 4.
Multi-head attention: Running several attention operations side by side on smaller slices of the vectors, so each head can track a different kind of relation. See chapter 18.
nDCG: Normalised discounted cumulative gain: relevance grades summed with a discount of log2(rank + 1), divided by the best possible ordering's sum. Rewards putting the most relevant results first. See chapter 26.
Neighbouring text: The end of the chunk before a retrieved passage and the start of the chunk after it, from the same document, added so the passage reads in context. See chapter 24.
Neuron: The basic unit of a neural network: a weighted sum of inputs plus a bias, passed through an activation function. See chapter 12.
Next-token prediction: The training task of a language model: given the text so far, give a probability to every possible next token. See chapter 16.
NF4: A 4-bit format whose 16 levels are spaced to suit bell-shaped weights, introduced with QLoRA (2023). See chapter 39.
Norm (length): The length of a vector: the square root of the sum of its squared entries. See chapter 4.
Normalisation: Rescaling numbers to a standard size: dividing a vector by its length, or shifting and scaling data to mean 0 and variance 1. See chapter 8.
Ollama: An open-source tool for running open-weight models on one machine, with a local API. The labs can use it instead of a paid model. See chapter 38 and Lab 0.
OpenRouter: A service that gives one API for many model providers, with routing rules per request. See chapter 38.
OpenTelemetry: An open standard for traces and spans. Many tracing tools accept spans in this format. See chapter 45.
Orchestrator: The code, or a lead agent, that runs the steps of a task in order, decides what comes next and hands work to workers or subagents. See chapter 29 and chapter 32.
Output check: The checks every answer passes before it is shown: deterministic rules first, then, if wanted, a model reviewer. See chapter 34.
Overfitting: When a model learns its training data so closely, noise included, that it does worse on new data. See chapter 11.
PagedAttention: vLLM's way of storing the attention key and value cache in fixed-size pages, like an operating system's virtual memory, so many requests share a GPU with little waste. See chapter 38.
PageIndex: A library that turns a long document into a tree of its sections. At question time a model reads the tree and picks the sections to open, instead of searching by vector similarity. See chapter 24.
Parameter (weight): A number inside a model that training adjusts. A "7B" model has about seven billion of them. See chapter 12.
Partial derivative: The slope of a function of several inputs along one input, holding the others fixed. See chapter 5.
Passages: Pieces of documents that retrieval puts into the prompt before the model reasons, marked as data and cited by document name and section. See chapter 24.
percentile (p50, p95): The value below which a given share of measurements fall: p50 is the median, p95 the value 95% stay under. See chapter 42.
Perplexity: e raised to the average cross-entropy per token. Roughly "how many tokens the model is choosing between". Lower is better. See chapter 19.
pgvector: A Postgres extension that stores vectors and finds the nearest ones. See chapter 24.
PII: Personally identifiable information: names, email addresses, phone numbers, ID numbers, bank account numbers, IBANs and addresses. See chapter 34 and Lab 7.
Placeholder: A stand-in token such as <PERSON_A>, <EMAIL_A> or <ACCOUNT_A> that replaces a piece of personal data while the text is outside your system. After Z the letters go on as AA, AB and so on. See chapter 34.
Pooling: Turning a transformer's one-vector-per-token output into one vector for the whole text, for example by averaging or by taking a special first token. See chapter 23.
Positional encoding: Extra numbers added to (or rotations applied to) token vectors so the model knows where each token sits in the sequence. See chapter 17.
precision and recall: For a check that flags items: recall is the share of truly bad items it flags; precision is the share of its flags that were truly bad. See chapter 34.
Precision@k: The share of the top k results that are relevant. See chapter 26 and Lab 4.
Presidio: An open-source library that finds personal data in text with recognisers and a spaCy language model. A scrubber can be built on it. See chapter 34.
Probability: A number from 0 to 1 that says how likely something is. See chapter 7.
Progressive disclosure: Loading information in stages, only as it is needed. A skill's description is always in context; its body loads when it is triggered; its bundled files only when read. See chapter 31.
Prompt: The text sent to a model. Usually a system prompt with standing instructions, earlier messages, and the new user message.
Prompt caching: A provider feature that stores the processed start of a long prompt so a repeat call is cheaper and faster. You mark the stable part of the prompt, such as the instructions and tool descriptions, for caching. See chapter 38.
Prompt injection: Text that tries to give a model new instructions, either typed by a user (direct) or hidden in a document or web page the model reads (indirect). See chapter 35 and Lab 7.
Prompt version: A label such as support_v5 naming one version of a prompt. Cost rows and traces carry it so results can be compared across prompt changes. See chapter 21 and Lab 2.
Provenance: Where a stored fact came from. A memory fact might be said (extracted from chat) or confirmed (the user pressed confirm). See chapter 37.
Provider routing: Gateway settings that pick which providers may serve a call, in which order, and whether they must keep no data. See chapter 38.
Pydantic: A Python library for typed data models with validation. It is often used to describe structured outputs and tool inputs. See chapter 28.
QLoRA: LoRA on top of a base model stored in 4 bits, so a large model can be fine-tuned on one modest GPU. See chapter 40.
Quantisation: Storing a model's numbers with fewer bits, such as int8 instead of fp32, to save memory and time at a small cost in accuracy. See chapter 39.
RAG (retrieval-augmented generation): Find relevant passages first, then give them to the model with the question so it answers from them. See chapter 24 and Lab 3.
Rate limit: A cap on requests per user per time window, such as 30 a minute. See chapter 34.
ReAct: A pattern where a model alternates reasoning, an action (a tool call) and an observation (the result) until it can answer. See chapter 29.
Reasoning tokens: Tokens a model writes to think before its answer. They cost like output tokens and count towards the output limit, so a small max_tokens can be used up before any answer appears. The effort setting controls how many are used. See chapter 16 and Lab 1.
Recall@k: The share of all relevant items that appear in the top k results. For an approximate index, the share of the true k nearest neighbours it returns. See chapter 26 and Lab 4.
Reciprocal rank fusion (RRF): Merging ranked lists by giving each item 1 / (60 + rank) from each list and adding them up. Uses ranks, not scores, so lists on different scales can be merged. See chapter 25 and Lab 4.
Regularisation: Anything that discourages a model from fitting noise, such as a penalty on large weights or dropout. See chapter 11.
ReLU: Rectified linear unit: max(0, x). The activation that made deep networks practical. See chapter 12.
Repeated tool call: A tool call with the same arguments as an earlier one in the same loop. A harness can answer it with the earlier result and a note to answer now, so the tool runs once. See chapter 28.
Reranker: A cross-encoder that scores each candidate passage against the question after a fast search, to reorder a short list such as the 50 nearest chunks. See chapter 25.
Residual connection: Adding a layer's input to its output, so the layer only learns a correction and gradients flow straight through. See chapter 13.
Retrieval: Finding the stored passages most relevant to a question. See chapter 24.
RLHF: Reinforcement learning from human feedback: train a reward model on people's preferences between answers, then tune the language model to score well on it. See chapter 19.
RNN: Recurrent neural network: reads a sequence one step at a time, carrying a hidden state forward. See chapter 14.
Rounding half to even: Rounding a value exactly halfway between two neighbours to the even one, so halves go down as often as up and totals do not drift. Python's built-in round does this: round(2.5) is 2 and round(3.5) is 4.
Rubric: A list of criteria with what earns each score, for example 0 to 2 per criterion. A judge scores an answer against it. See chapter 44.
rule of three: If none of n checked items failed, the true failure rate is below about 3/n with 95% confidence. See chapter 36.
Sampling: Choosing the next token at random according to the model's probabilities, instead of always taking the most likely one. See chapter 16.
scale (quantisation): The number that turns stored integers back into real values: real value = integer × scale. Set by the largest weight in its group. See chapter 39.
Schema repair: When a model's structured output fails validation, ask once more with the error and the earlier exchange. See chapter 28.
Seeded defect: A known flaw planted into a good answer on purpose, to test whether a judge catches it. See chapter 44.
SFT: Supervised fine-tuning: train a model on example prompts with the answers you want. See chapter 19.
Sigmoid: The S-shaped function 1 / (1 + e^-x) that squeezes any number into the range 0 to 1. See chapter 10.
Skill: A folder with a SKILL.md file (a name, a description, then instructions) and optional scripts and reference files, which an agent loads only when a task needs it. See chapter 31.
soft targets: A teacher model's full probability list, often softened by a temperature, used to train a student in distillation. See chapter 40.
Softmax: Turns a list of scores into probabilities: raise e to each score, then divide each by the total. See chapter 9.
Span: One timed step inside a trace, such as one model call. See chapter 45 and Lab 9.
Spend guard: Code that refuses a paid model call when it could push spending past a set limit. See chapter 45.
SSE (server-sent events): A simple way for a server to stream messages to a browser over one HTTP response. Chat services stream an answer this way as it is written, and MCP's Streamable HTTP can use it. See chapter 30.
SSRF (server-side request forgery): Tricking a server into fetching an address of the attacker's choosing, often an internal one. Fetch tools guard against it by allowing only public hosts. See chapter 35.
Standard deviation: The square root of the variance: a typical distance of a value from the mean, in the data's own units. See chapter 8.
stdio transport: Running a local MCP server as a child process and exchanging one JSON message per line over its standard input and output. See chapter 30 and Lab 6.
Streamable HTTP: MCP's HTTP transport: one endpoint that accepts POST and answers with JSON or a stream of server-sent events. A local server binds to 127.0.0.1 and checks the Origin header. See chapter 30.
Structured output: Asking a model for JSON that matches a schema, and validating it, instead of free text. See chapter 16.
Subagent: A separate agent started for one self-contained task, with its own instructions, tools and empty context window, which returns only a report. See chapter 32.
System prompt: The standing instructions sent before the conversation. Keeping them in their own file makes changes easy to track. See chapter 21.
Temperature: A number that divides the logits before softmax. Below 1 sharpens the distribution; above 1 flattens it. See chapter 16.
Tensor: A grid of numbers with any number of axes: a vector has one, a matrix two, a batch of images four.
Test set: Data kept aside and used once, at the end, to estimate how a model does on new data. See chapter 11.
time to first token: How long a model call takes to produce its first output token; streaming shows the answer from this point. See chapter 42.
Token: A piece of text, often part of a word, that a language model reads and writes as one unit. See chapter 16.
Token passthrough: A server accepting an access token that was not issued to it and forwarding it to another service. Forbidden by the MCP specification, because it bypasses the other service's checks and breaks the audit trail. See chapter 30.
Tokeniser: The fixed procedure that splits text into tokens and maps each to an integer id. See chapter 16.
Tool (function calling): A function the model may ask the program to run, described by a name, a purpose and an input schema. The program runs it and returns the result. See chapter 28 and Lab 5.
Tool call: A request from the model, returned in its reply, to run one named tool with given arguments. It carries an id so the program can match the result it sends back. The model never runs anything itself. See chapter 28 and Lab 5.
Tool argument error: A tool call whose arguments fail the tool's schema. Send it back to the model as an error result naming the fields, never their values, so the model can correct the call instead of the call ending. See chapter 28 and Lab 5.
Tool loop: The back-and-forth in one model call: the model asks for tools, the program runs them and sends results back, until the model answers. See chapter 28 and Lab 5.
Tool trace: The list of tool calls an agent made, with arguments and results. Requiring every figure in an answer to point into it catches invented numbers. See chapter 28.
Top-k: Keep only the k best-scoring items. A retriever returns the top k passages, for example the top 3 or top 5. See chapter 24.
Top-p (nucleus sampling): Sample the next token only from the smallest set of tokens whose probabilities add up to at least p. See chapter 16.
Trace: In observability, the record of one request as a tree of timed spans. See chapter 45 and Lab 9.
traceability: Recording enough with each answer (inputs, sources, versions, checks) to rebuild how it was produced. See chapter 43.
Transformer: The neural network design behind today's language models: stacked blocks of attention and MLP layers with residual connections. See chapter 18.
Typed reference: An answer segment that names a tool, a field path into its result and a format, instead of writing a number. Code fills the number in, so the model cannot misquote it. See chapter 28.
Underfitting: When a model is too simple to capture the pattern, so it does badly on training and new data alike. See chapter 11.
uv: A fast Python package and project manager. Commands run as uv run ..., and dependencies can be locked in a uv.lock file.
Validation set: Data kept out of training and used to choose hyperparameters and to notice overfitting. See chapter 11.
Vanishing gradient: When gradients shrink as they pass back through many layers, so early layers barely learn. See chapter 14.
Variance: The average squared distance of values from their mean: a measure of spread. See chapter 8.
Vector: An ordered list of numbers, which can be drawn as an arrow from the origin. See chapter 4.
vLLM: An open-source engine for serving language models at high throughput, using PagedAttention and continuous batching. See chapter 38.
Weight: A learned number that multiplies an input inside a model. "Weights" often means all of a model's parameters. See chapter 12.
Wilson interval: A confidence interval for a proportion that stays between 0 and 1 and behaves well for small counts, unlike the plus-or-minus formula. 3 failures in 50 gives about 2.1% to 16.2%. See chapter 36.
Workflow: A design in which code fixes the steps and their order and models fill in some of them, as opposed to an agent, where the model chooses its next step. See chapter 29.
XOR: "Exclusive or": true when exactly one of two inputs is true. No single straight line separates its cases, which showed the limit of one-layer networks. See chapter 14.
ZDR (zero data retention): A provider promise to keep no copy of prompts or outputs. Some gateways let you limit calls to providers that make it. See chapter 38.
Zero point: In affine quantisation, the integer that stands for the real number 0. Together with the scale it maps real values to integers. See chapter 39.
z-score: How many standard deviations a value is from the mean: (x − mean) / standard deviation. See chapter 8.

Glossary

A

B

C

D

E

F

G

H

I

J

K

L

M

N

O

P

Q

R

S

T

U

V

W

X

Z