Part V · Build it

Lab 1: your first model call

Send one message to a language model and read everything that comes back: the words, why the model stopped, how many tokens it read and wrote, how long it took and what it cost. Then give it standing instructions, hold a conversation, and see why every turn of a conversation costs more than the one before.

What you will build

A command-line tool, labs/01-first-call/first_call.py, of about 230 lines, and its tests in test_first_call.py. It can:

• send one message and print every field of the reply;
• send a system prompt beside the message, and show the request before it goes;
• hold a conversation (--chat), printing tokens and cost after each turn;
• set --max-tokens, --effort (Claude) and --temperature (Ollama).

Before you start

Finish Lab 0. Any backend works. The outputs on this page come from the fake backend, so you can follow along for free and compare your screen with the page:

export LLM_BACKEND=fake

Then run each step again on Claude or Ollama, where the replies are real and will differ from run to run.

Step 1: send one message

A model call is a function call over the network. You send a list of messages; the model reads them as tokens, writes its reply one token at a time and sends it back (chapter 16). With the shared adapter from Lab 0, a one-message conversation is a list of one dictionary:

def ask(
    prompt: str,
    *,
    system: str | None = None,
    max_tokens: int = DEFAULT_MAX_TOKENS,
    effort: str | None = None,
    temperature: float | None = None,
) -> llm.Reply:
    """Send one user message and return the reply. The whole conversation is one turn."""
    messages = [{"role": "user", "content": prompt}]
    return llm.chat(messages, system=system, max_tokens=max_tokens, effort=effort,
                    temperature=temperature)

Run it:

$ python labs/01-first-call/first_call.py "Say hello in five words"
reply        [fake model] I received 1 message; no system prompt. You said: Say hello in five words
stop_reason  end
tokens       14 in, 21 out
latency      0.000 s
model        fake (fake backend)
cost         $0
             the same tokens on claude-haiku-5-5 would cost $0.000012

The fake backend repeats what it received, which proves the message travelled. On Claude the reply line is a real greeting.

Step 2: read the reply

chat() returns a Reply, the same shape whichever backend answered. Each field answers a question you will ask of every call you ever make:

The extra line under cost is this lab's own: what the same token counts would cost on claude-haiku-5-5, whatever answered. On the fake backend the counts are estimates, about four characters per token; a real tokeniser counts differently, and Claude adds its thinking to the output (chapter 16). So read the fake's figure as a sketch of the order of magnitude, not a quote.

Latency on the fake backend is close to zero because nothing leaves your computer. On a real model, most of the wait for a long answer is the writing, one token at a time, which is why streaming the answer as it is written feels faster (chapter 42).

Step 3: add a system prompt

A system prompt holds standing instructions: who the model is working for, the rules and the shape of the answer (chapter 21). It travels beside the conversation, not inside it, and applies to every turn. --show-request prints what is sent before sending it:

$ python labs/01-first-call/first_call.py --show-request --system "Answer in one sentence." "What is a token?"
request {
  "system": "Answer in one sentence.",
  "max_tokens": 1024,
  "messages": [
    {
      "role": "user",
      "content": "What is a token?"
    }
  ]
}
reply        [fake model] I received 1 message; system prompt: "Answer in one sentence." You said: What is a token?
stop_reason  end
tokens       18 in, 25 out
latency      0.000 s
model        fake (fake backend)
cost         $0
             the same tokens on claude-haiku-5-5 would cost $0.000014

Two things to notice. The messages list still holds only your question; the instruction sits in its own field. And the input is 18 tokens, where the same question without --system counts 12 on the fake backend: the system prompt is sent, and paid for, on every call. A long system prompt on a busy service is a real cost, which is what prompt caching addresses (chapter 22).

Step 4: hold a conversation

A model remembers nothing between calls. A conversation is a list that you keep: you append the user's turn, send the whole list, and append the model's reply before the next turn.

def send(self, text: str) -> llm.Reply:
    self.messages.append({"role": "user", "content": text})
    # Send a copy, so a record of this request is not changed by later turns.
    reply = llm.chat(list(self.messages), system=self.system, max_tokens=self.max_tokens,
                     effort=self.effort, temperature=self.temperature)
    self.messages.append(reply.message())
    self.turns += 1
    self.input_tokens += reply.input_tokens
    self.output_tokens += reply.output_tokens
    self.cost_usd += reply.cost_usd
    return reply

reply.message() gives the assistant's turn in the shared format. On Claude it also carries the provider's own content blocks, including thinking, which must go back unchanged on the next call; the adapter does that for you. Hold a three-turn chat:

$ python labs/01-first-call/first_call.py --chat --system "You are a friendly travel planner."
Chatting with fake (fake backend). Empty line or /quit to stop, /reset to start again, /history to see what is sent.
you> I am planning a three-day cycling trip.
model> [fake model] I received 1 message; system prompt: "You are a friendly travel planner." You said: I am planning a three-day cycling trip.
(turn 1: 26 in + 34 out tokens, 0.000 s, $0; so far 60 tokens, $0; on claude-haiku-5-5 about $0.000020)
you> What should I pack?
model> [fake model] I received 3 messages; system prompt: "You are a friendly travel planner." You said: What should I pack?
(turn 2: 83 in + 29 out tokens, 0.000 s, $0; so far 172 tokens, $0; on claude-haiku-5-5 about $0.000042)
you> What did I say I was planning?
model> [fake model] I received 5 messages; system prompt: "You are a friendly travel planner." You said: What did I say I was planning?
(turn 3: 138 in + 32 out tokens, 0.000 s, $0; so far 342 tokens, $0; on claude-haiku-5-5 about $0.000072)
you> /quit

The fake model counts 1, 3 and 5 messages: each call carries every earlier turn. That is why the input tokens climb from 26 to 83 to 138 although each question was short. With real answers, which are longer, the climb is steeper. Worked through: if every turn adds 100 tokens to the history, turn n sends about 100n input tokens, so ten turns send 100 + 200 + … + 1,000 = 5,500 input tokens in all, not 1,000. The cost of a conversation grows with the square of its length, and a long one eventually fills the context window (chapter 22). In the chat, /history prints the list being sent and /reset starts again.

Step 5: when max_tokens is too small

max_tokens is a hard ceiling on the tokens the model may write. The model does not plan around it: when the ceiling is reached, the reply simply stops, mid-word if need be, and stop_reason says max_tokens. The fake backend imitates that, so you can see the code path:

$ python labs/01-first-call/first_call.py --max-tokens 8 "Explain tokens in three paragraphs"
reply        [fake model] I received 1 messag
stop_reason  max_tokens
tokens       16 in, 8 out
latency      0.000 s
model        fake (fake backend)
cost         $0
             the same tokens on claude-haiku-5-5 would cost $0.000006
note         the reply hit max_tokens and was cut off; raise --max-tokens. On Claude, thinking counts toward max_tokens, so the text can even be empty.

On Claude there is a second trap. The model thinks before it answers, and the thinking counts toward max_tokens (chapter 16). With a small limit the whole budget can go on thinking, and the reply comes back with empty text and stop_reason max_tokens. Three habits follow:

• Check stop_reason before you use text. A cut-off answer can look complete.
• Leave room. This lab's default is 1,024 and the shared adapter's is 2,048. Control the length of the answer with instructions ("in one sentence"), not with the ceiling.
• On Claude, use --effort low for simple tasks: less thinking, so fewer output tokens, a lower cost and a faster reply.

Step 6: compare backends

The same command runs on every backend; only the environment changes:

LLM_BACKEND=ollama python labs/01-first-call/first_call.py "Say hello in five words"
LLM_BACKEND=anthropic python labs/01-first-call/first_call.py "Say hello in five words"
LLM_BACKEND=anthropic LLM_MODEL=claude-sonnet-5-5 python labs/01-first-call/first_call.py "Say hello in five words"

Real replies vary, so there is no output to copy here. Compare four things:

• The words. Did each model keep to five words? Small models often do not.
• The token counts. Each model family has its own tokeniser, so the same sentence counts differently, and Claude's output includes its thinking.
• The latency. A local model's speed depends on your hardware; a hosted model's includes the network and the thinking.
• The cost. Sonnet 5.5 is 20 times Haiku 5.5's price per token. Whether a stronger model is worth it depends on the task; chapter 22 covers how to choose, and Lab 2 builds the tool that measures it.

--temperature changes Ollama's answers (chapter 16). Claude ignores it, because current Claude models fix their own sampling and reject a temperature, so the adapter leaves it out.

Checklist

• I sent one message and read the reply.
• I can say what each field of the reply means.
• I added a system prompt and saw where it goes in the request.
• I held a three-turn chat and watched the input tokens grow.
• I made a reply stop at max_tokens.
• I ran the same prompt on two backends and compared them.
• The lab's tests pass.

Run it

The tests check the offline logic with the fake backend: the request carries the system prompt in its own field, a conversation resends its whole history, a small max_tokens stops a reply, and the costs use Haiku's prices. Run them from labs/:

$ python -m pytest 01-first-call
.............                                                            [100%]
13 passed in 0.02s

Then the whole lab, in order, from the repository root:

python labs/01-first-call/first_call.py "Say hello in five words"
python labs/01-first-call/first_call.py --show-request --system "Answer in one sentence." "What is a token?"
python labs/01-first-call/first_call.py --chat --system "You are a friendly travel planner."
python labs/01-first-call/first_call.py --max-tokens 8 "Explain tokens in three paragraphs"

What it costs. Nothing on Ollama or the fake backend. On claude-haiku-5-5, the steps make about 20 calls. Allowing each call up to 1,000 input tokens (the chat's history grows) and 600 output tokens (an answer plus its thinking), the lab costs at most 20 × (1,000 × $0.10 + 600 × $0.50) / 1,000,000 = $0.008, under one US cent. Your own runs print their real cost after every call.

Experiments

1. Temperature, on Ollama. Ask "Name a colour." three times with --temperature 0, then three times with --temperature 1.5. How much do the answers vary? Why might you want each setting?
2. Effort, on Claude. Ask a small puzzle, such as "A bat and a ball cost 1.10 in total. The bat costs 1.00 more than the ball. How much is the ball?", with --effort low and then --effort high. Compare output tokens, latency and whether the answer is right.
3. Find the cliff. On Claude, ask for a three-paragraph explanation with --max-tokens 16, 64, 256 and 1024. At which limits does the text come back empty, cut off or whole?
4. Count tokens. Send the same sentence in English and in another language, then a line of code and a long number. Compare the input token counts (chapter 16).
5. A long chat. Hold ten turns and write down the input tokens for each. Do they grow as step 4 predicts? Then type /reset and ask a follow-up question: what does the model know now?
6. Where instructions live. Put "Answer in French." in the system prompt for one chat, and in the first user message for another. Which one does the model still follow after five turns?

Going further

• Stream the answer. Print tokens as they arrive instead of waiting for the whole reply, and time the first one (time to first token). The Claude SDK's streaming guide is in the further reading.
• Retry politely. Wrap chat() so that it waits and tries again after a rate-limit error, a little longer each time (chapter 42).
• Save a conversation. Write messages to a JSON file when the chat ends and load it at the start of the next. That is the first step towards memory (chapter 37).
• Measure, do not guess. Lab 2 turns single calls into a test bench that scores a prompt over thirty cases.