Part IV · Applied concepts · Chapter 28

Tools and function calling

A language model can only write text. It cannot read a database, check a price or do exact arithmetic. Tool calling (also called function calling) closes that gap: the model writes a small structured request, your code runs it, and the result goes back to the model as its next input. Almost everything useful an AI application does passes through this one mechanism.

What a tool call is

When you call a model with tools, you send three extra things with the conversation: each tool's name, a description of when to use it, and an input schema saying what arguments it takes. The model can then reply in one of two ways: with ordinary text (its answer), or with a tool request that names a tool and gives arguments that fit its schema. A tool request is not an action. Nothing happens until your code reads it, decides to run it, and sends back the result.

The model learned this request format in post-training (chapter 19): it has seen many examples of tool definitions, requests and results, so it knows the pattern. It still chooses, by predicting tokens, whether to call a tool, which one, and with what arguments. Every one of those choices can be wrong.

Describing a tool: names, descriptions, schemas

The input schema is written in JSON Schema, a standard way to describe the shape of JSON. This is the schema the Python MCP SDK generated for the compound_value tool in the project of chapter 30, as it went over the wire:

The schema was built from the Python function's type hints, and the description from its docstring. Money is a string, not a number, on purpose: a JSON number may be read as a binary floating-point value, and 0.1 cannot be stored exactly in binary, so decimal strings keep cents exact.

Good tool design is mostly good naming and narrow scope:

• Name and describe for the model. The description is the only manual the model gets. Say what the tool returns and when to use it, not how it works inside.
• Narrow and typed. get_order(order_id) is better than run_sql(query): the model cannot ask for something you did not mean to expose, and bad arguments fail at the schema.
• Read-only by default. A tool that only reads cannot do damage when the model calls it at the wrong moment. Anything that writes, pays or sends needs a person's confirmation outside the model.
• State the limits. A description should say where the data stops: "the shop's stock, not the warehouse's"; "a null value is unknown, not zero". Accept an identifier in the forms people use (an id or a code, in any case), and record a failed call in the trace as well as a successful one.
• Bind per request. The user's identity and the trace recorder belong to one request. Pass them through request-scoped context, not globals, so tools running at the same time cannot read each other's data or trace.
• Few at a time. Every tool definition costs tokens on every call, and the more similar tools there are, the more often the model picks the wrong one. Offer each step only the tools it needs.
• Errors the model can act on. "years must be between 0 and 50" lets the model fix its call; "internal error" does not. Return the error as a result instead of crashing the loop.
• Results marked as data. A tool result may contain text from a document or web page. Label it as data, never instructions (chapter 35).

The tool loop and its cost

One question often needs several tools in sequence: read the order, then the stock level, then compute. The loop is: call the model; if it asks for tools, run them, append the requests and results to the conversation, and call it again; stop when it answers in text. Because a model keeps no memory between calls, each call resends the whole conversation so far.

Worked: the first call sends 3,000 tokens (instructions, tool definitions, question). Each tool round adds a 50-token request and a 500-token result. With four tool rounds before the answer, the five calls send 3,000, 3,550, 4,100, 4,650 and 5,200 tokens: 20,500 input tokens for one answer, almost seven times the first call. That arithmetic series is why tool results should be small, and why a cap on the number of rounds is also a cap on cost.

Every loop needs a turn limit. A model can get stuck calling the same tool again, or wander. When the limit is hit, the loop stops instead of running forever, and your code decides what the user sees. One useful choice is a last call with no tools at all: the model has to answer from what it has already read.

Two smaller guards keep a loop from ending badly. A repeated call, the same tool with the same arguments, can be answered with the earlier result and a note to answer now, so the tool does not run twice and the model is pushed on. And an argument that fails the schema can go back to the model as an error result, naming the field and what was wrong, instead of ending the call: the model fixes its call, as it would after any other error result. Name the field and the kind of error, but do not echo the value: a validation message can quote private details.

Fencing a tool that reaches outside

A tool that searches or fetches from the web is the riskiest kind, because it brings untrusted text into the conversation. Fence it:

• An allow-list of hosts. A search hit off the list is dropped before anything is fetched, and a fetch of any other address is refused.
• A budget. For example 2 searches, 3 fetches and 20 seconds per lookup. When a budget is spent, the tool says so in words the model can act on.
• Quotes must be on the page. Keep a quote only when its text appears in a page the tool really fetched, and return each quote with its URL, the page title and the date it was read.
• Fail in the open. If the lookup fails, the model reads "web lookup is unavailable" and answers without it, instead of guessing what the page would have said.

Structured output and one repair

The final answer can be constrained too: you send an output schema and ask the model to reply with matching JSON. Code then validates the reply. Models follow schemas well but not perfectly, so a common pattern is validate, then repair once: if the reply does not validate, send the validation error back with the conversation and ask for corrected JSON only. If the second reply also fails, give up cleanly. One repair fixes most slips; a loop of repairs mostly burns money on a model that has misunderstood the task.

Let code own the numbers

A model that copies a figure from a tool result into its prose can mistype it, round it, or invent one that was never there, and a reader cannot tell. A stronger design keeps digits out of the model's prose entirely: wherever a figure belongs, the model writes a typed reference, naming a tool and a path to a field in that tool's result, and code fills in the value. Code can then check every reference: the tool was really called, the field exists, the source is dated. The same idea runs through the formula tool of chapter 27, where the model writes formulas over named inputs and code does the arithmetic.

Go one step further and let code choose the amount as well. A tool can compute a whole proposal, for example the size of a refund, or the one item to swap to close a gap, together with the before and after values; the model never picks the number, it explains the one the tool returned. Answers may list proposed actions, but draw them as descriptions or as buttons that only ask a person. Nothing a model returns should move money, send a message or delete data by itself.

Calculators: exact maths in code

Arithmetic is the clearest case for a tool. A model predicts digits as tokens and can slip on long multiplications; code cannot. Worked with the future-value formula used for savings: a lump sum of 10,000 plus 500 a month, at 6% a year compounded monthly, for 10 years. The monthly rate is 0.06 / 12 = 0.005 over 120 months, so the growth factor is 1.005¹²⁰ = 1.819397.

• The lump sum grows to 10,000 × 1.819397 = 18,193.97.
• The contributions grow to 500 × (1.819397 − 1) / 0.005 = 81,939.67, the sum of a geometric series (each month's 500 grows for one month fewer than the one before).
• Total: 100,133.64.

Asking a model to produce 100,133.64 by itself invites an answer that is close but wrong. Asking it to call a calculator with four inputs, each of which it can read off the question, makes the number exact and checkable.