Skip to content

Test with ScriptedModel

ScriptedModel plays back the answers you script, one for each model call. You use it to test an agent on your own machine, with no API key, and to get the same result every run.

This test runs the order agent against two scripted answers. The first calls lookup_order and the second replies. It checks that the script ran out, then returns the agent’s answer:

offline-order-agent.ts
/** Runs the order agent on scripted answers, checks the script ran out, and returns the agent's answer. */
export async function answerOffline(): Promise<string> {
const adapter = new ScriptedModel([
{ content: [{ type: "tool_call", callId: "call-1", name: "lookup_order", input: { order: ORDER } }] },
{ content: [{ type: "text", text: `Order ${ORDER} has shipped.` }] },
]);
const model = createModel({ adapter, id: "scripted-model" });
const findOrder = async (id: number) => ({ id, status: "shipped" as const });
const answer = await answerOrderQuestion(model, findOrder, `Where is order ${ORDER}?`, () => {});
adapter.assertComplete();
return answer;
}

A strict model fails when your test drifts

Section titled “A strict model fails when your test drifts”
  • Too many calls throw. A call past the last scripted answer throws, and the gateway turns the throw into a failed call, so the run ends with the stop reason error.
  • assertComplete() catches both drifts. It throws if an answer was never used, or if a call came after the last answer. Call it when the test ends.
  • Requests are snapshots. The model’s calls holds a frozen copy of each request, so later changes to your conversation never show up in it.
  • Streams look real. Text and tool-call input arrive in small pieces, so your streaming code runs as it would against a provider.

A scripted answer can also be a function that builds the response from the request, which lets a test read the conversation the model was sent.

Next, keep every test offline.

These docs are licensedCC BY 4.0. Thelicence text is in the repository.