Part IV · Applied concepts · Chapter 34

Guardrails

A guardrail is a rule, enforced by code, that sits between a model and the world: it decides what may go into the model, what the model may do, and what may come out of it to a customer. A model's good behaviour is a tendency; a guardrail turns the behaviours you cannot do without into guarantees, or at least into things you can measure.

Three places to put a guardrail

Put the cheap, exact checks first. A length check costs nothing and is never wrong about the length; a model classifier costs money and is sometimes wrong. If the cheap check already says no, the expensive one never runs.

Input guardrails

Caps and rate limits. A cap on input length bounds the cost of any one request. A rate limit bounds how fast one user can spend. A common form is a sliding window: keep the times of a user's recent requests, drop those older than the window, and refuse when the count left is at the limit.

Worked: the limit is 30 requests per 60 seconds. A user has sent requests at seconds 0, 1, 2, … 29 (30 requests). At second 45 all 30 are still inside the window, so request 31 is refused. At second 61 the request from second 0 is more than 60 seconds old and is dropped, 29 remain, and the new request is allowed.

Topic and abuse classification. A small, cheap model reads the question and says whether it is on topic and whether it is abusive, before any expensive work starts. Its answer is a label in a fixed schema, not free text, so code can act on it.

Personal-data scrubbing. Names, phone numbers, emails, identity numbers and bank account numbers are replaced by placeholders before any text reaches a model, and the real values are put back only in the text the customer sees. The model can still say "thanks, <PERSON_A>" without ever knowing the name.

Order of the gate. Run the code limits first (input length, rate limit, a daily token budget per user and in total) and only then ask the classifier model. If the classifier errors, fail closed: send a fixed safe reply, not a guess. A question whose stakes the classifier cannot judge should count as high stakes. Counters kept in one process reset when it restarts and do not add up across servers; that is acceptable in a prototype if you write it down, and not in production.

What makes a scrubber hold up

A scrubber fails in two directions: it misses personal data, and it masks things that are not personal. Most of its quality comes from small rules like these:

• Use context for numbers. A long digit string is a bank account number only after a word such as "account" or "a/c" and with enough digits (say 9), so a bare amount such as "120000" stays in the clear. An IBAN is recognised by its shape: two letters, two check digits, then 11 to 30 letters or digits.
• Money is not a phone number. A figure written with two decimal places ("800.00") is an amount, even though a phone-number library may read it as a toll-free number.
• Numbers with a country code are phone numbers even outside the ranges a phone library holds as valid; after "+" accept any code, but after "00" accept only the codes you serve, since account and order numbers also start with "00".
• End a name at the first word that holds a digit. In "Ravi Kumar 5000" the name is masked and the amount kept, and a sentence-opening word such as "Email" or "Call" is not taken for a name.
• Replace overlapping matches as one. An email address that contains a first name must not leave its tail in the clear.
• Label by kind, count like spreadsheet columns. <PERSON_A>, <EMAIL_A>, <IBAN_A>, then A to Z, then AA, AB and on, so a long text never runs out of labels. Keep one pattern for them all (<[A-Z]+_[A-Z]+>) so restoring is one pass.
• Mask dictionaries by key name. Data bound for logs and traces is masked by the names of its keys (anything ending in no, bank identifiers, the last four digits of an account), not by guessing at the values.

Where the real values come back. Scrub at the start of the request and again at the gateway (chapter 33). Real values come back only in what the person sees on screen, never in what the server stores, judges or logs. Facts recalled from an earlier request must not take this request's values: a person reads as "a person" and another placeholder reads as plain words for its kind, such as "an account number" (chapter 37).

Measure it. Build a test set of rows with planted personal data and rows of clean text. Require 100% recall for the formats you must never miss (emails, identity numbers, account numbers, IBANs, phone formats), a lower bar for names and addresses, and zero false masks on clean text. Also check each planted row after masking: fail if a planted value is left in the clear, if text is glued to the end of a placeholder (the tail of a value replaced short), or if words that must stay, such as an amount after a name, are lost.

Output guardrails

• Validate the shape. Ask for structured output and validate it against a schema. A model answer that does not parse is not an answer.
• Deterministic rules next. Anything code can decide exactly (every figure comes from a tool result, every source has a date, the required disclosure is the last line) is decided by code.
• A model check last, for what code cannot decide: is this suitable for this customer's risk profile? Is it safe? Give the checking model a rubric with levels, and a strict pass mark.
• Fail closed. If a check errors, the answer is not shown. A guardrail that passes everything when it breaks is worse than none, because people believe it is there.
• Cheap first, in code. Run the deterministic rules first and call the model check only if they pass. The model check scores each rubric criterion (for example suitability and safety) on a small scale, 0 to 2, and blocks on anything below the top mark.
• Allow exact exceptions. A rule such as "no digits in prose" can skip a headline quoted word for word from this answer's own tool results; a headline changed by one character is checked like any other prose.

Required wording, owned by code

Some lines must appear word for word: a disclosure that says an answer is AI-generated and not a guarantee, for example. A model asked to write such a line will sometimes paraphrase it, shorten it or forget it. So the wording lives in code as a constant, code decides which wording an answer needs, and a deterministic check refuses an answer whose last line of prose is not that exact wording. An answer that also cites a public web page ends with a second fixed sentence on that same line, and the check wants both.

Two refinements make this work in practice. First, when the right wording depends on what the answer used, code chooses it from the record of what happened, not from the model's own claim. Second, when the only fault is that the line is missing, code can add it, because the words are known exactly; any other fault still blocks the answer. The prompts render the same constant, so the model and the check never disagree about the words.

Worked: a medication-information assistant looks up the leaflet for a drug. If the lookup found a current leaflet, code picks the wording that names the manufacturer's leaflet and its date. If the lookup found none, code picks the wording that says no current leaflet was on file and to ask a pharmacist. If it found a leaflet and also cited a clinical guideline, code picks the wording that names both sources. The model wrote none of these choices; the tool trace decided each one.

How good is a guardrail? Precision and recall

A guardrail that blocks answers is a classifier, and it makes two kinds of mistake: it lets bad answers through (misses) and it blocks good ones (false alarms). Count both.

Worked: of 1,000 answers, 50 are bad. The check blocks 45 of the bad ones and 30 of the 950 good ones.

Two checks in a row: if a second check catches 80% of what the first one missed, and the two fail independently, the bad answers that get through fall from 5 to 5 × 0.2 = 1 per 1,000. False alarms add up the other way: every check you add can block good answers too. Independence is the weak assumption: two checks that look at the same thing tend to miss the same answers.

Guardrail tools

Libraries exist for each layer: Presidio finds and replaces personal data; NeMo Guardrails and Guardrails AI let you declare rules about topics, dialogue and output shape. They are useful, but each one still needs the measurements above on your own data before you trust it.