llm

import { llm } from '@routecraft/ai'

Call a language model and get text or structured output. Requires llmPlugin() in your context plugins.

import { llm } from '@routecraft/ai'

type Document = { content: string; text: string }

// Text output
craft()
  .id('summarise')
  .from<Document>(source)
  .enrich(llm('anthropic:claude-haiku-4-5-20251001', {
    system: 'Summarise the following in one sentence.',
    user: (ex) => ex.body.content,
  }))
  .to(log())
// Result replaces the body: { text: '...', usage: { inputTokens, outputTokens } }
// (use only() to merge instead, e.g. .enrich(llm(...), only((r) => r.text, 'summary')))

// Structured output with Zod schema
import { z } from 'zod'

const sentimentSchema = z.object({
  sentiment: z.enum(['positive', 'neutral', 'negative']),
  confidence: z.number(),
})

craft()
  .id('classify')
  .from<Document>(source)
  .enrich(llm('openai:gpt-4o', {
    system: 'Classify the sentiment of the text.',
    user: (ex) => ex.body.text,
    output: sentimentSchema,
  }))
  .to(log())
// result.output is typed as { sentiment: 'positive' | 'neutral' | 'negative', confidence: number }

Model ID format: "provider:model-name" (e.g., "ollama:llama3.2", "anthropic:claude-sonnet-4-6").

Supported providers: openai, anthropic, ollama, openrouter, gemini, lmstudio, custom

Options:

Result shape (replaces the body in bare .enrich() / .to(); pass an aggregator such as only() to merge):

Provider credentials are configured once in llmPlugin() and shared across all llm() calls. See Plugins reference.

Content parts

A user prompt can be an array of content parts instead of a string, so a route that receives a voice note or an image hands it to the model directly rather than transcribing it first.

craft()
  .id('answer-voice-note')
  .from<{ audio: string; caption: string }>(source)
  .enrich(llm('gemini:gemini-3-flash-preview', {
    user: (ex) => [
      { type: 'file', data: ex.body.audio, mediaType: 'audio/ogg' },
      { type: 'text', text: 'Answer the question in the recording.' },
    ],
  }))
  .to(log())

data and image take bytes (Uint8Array, ArrayBuffer), a base64 string, or a URL.

system stays string-only. No provider accepts content parts there.

An empty parts array reads as "the author said nothing", exactly as an empty string does. The two destinations then differ, as they already did for strings: llm() falls back to the body-derived prompt, while agent() sends the empty prompt as given. A callback that maps attachments to parts returns [] on an exchange that carries none, so give it an explicit fallback rather than relying on the destination's.

Which providers accept what is theirs to say, not ours. Nothing is pre-validated per provider: the parts are forwarded, and a provider that cannot read one fails through its own error rather than a framework guard that would go stale. The AI SDK's own prompts documentation is the current list, and at the time of writing names Google Generative AI, Google Vertex AI, Anthropic, and OpenAI (PDFs, plus wav and mp3 on the audio-preview models) as the providers taking file parts. Image parts are broadly supported. Support is per model as well as per provider, so check the model you are actually calling.

Two behaviours worth knowing before you reach for a URL:

• A URL part is downloaded by the SDK unless the provider declares it can fetch it itself. The bytes then travel in the request, from your process. A provider that advertises URL support gets the URL verbatim instead. The framework does not bound that fetch: no size cap, no timeout of its own, no scheme allow-list, unlike the http() client which bounds the responses it accepts. A URL derived from an inbound body is therefore an outbound request a stranger chose, and deciding whether a URL is acceptable belongs in the route, where a reader can see it.
• Neither raw bytes nor a URL object can cross a deferral boundary. An agent that defers (a deferring tool, a human approval) persists its thread as JSON data, so a Uint8Array or a URL instance in a part is refused at deferral time with a message naming the exact part. On any route that can defer, pass a base64 string, or the URL as a plain string: the SDK reads a URL-shaped string as a URL, and a string survives the store. See deferral.

Reasoning effort

Reasoning effort is the main cost and latency dial on a current model, and it belongs per call rather than per deployment: a narrow classifier and a hard agent turn want opposite settings against the same model id.

craft()
  .id('classify')
  .from(source)
  .enrich(llm('gemini:gemini-3-flash-preview', {
    system: 'Answer yes or no.',
    reasoning: 'none',
  }))
  .to(log())

reasoning is normalised, so the same route survives a provider swap. Each level maps to the provider's own control:

A level a provider cannot express maps to the nearest one it supports rather than throwing, because an option that refuses on some providers is not portable and portability is the point. Three rows are lossy, and one maps nothing at all:

• Gemini's thinkingLevel has no off position. none reaches it as minimal, its lowest level. A Gemini model still thinks a little.
• Ollama's control is a boolean. low, medium and high are one value there.
• LM Studio forwards the string to whatever server is behind it, so the level only means something when the loaded model reads it.
• custom maps nothing. The model handle is yours, so the framework cannot know which namespace its settings belong under. Use providerOptions.

Mapping is where the framework's part ends. Each level is forwarded as the provider's own reasoning-effort setting, and which values a given model accepts is the provider's contract rather than ours: a model that will not take the value it is sent refuses at the API rather than at the call site. Anthropic's effort is the clearest case, sent as an output_config field that needs a model new enough to accept one. The same holds for none where it is forwarded verbatim. The table says what is sent, not what every model behind a provider id does with it.

Leave reasoning unset and nothing is sent, so the provider's own default applies.

Reaching a provider directly

providerOptions is forwarded to the SDK verbatim, keyed by the SDK's provider namespace. Use it for anything the normalised options cannot say, such as Anthropic's thinking token budget:

.enrich(llm('anthropic:claude-sonnet-4-6', {
  providerOptions: {
    anthropic: { thinking: { type: 'enabled', budgetTokens: 8192 } },
  },
}))

The namespace is the SDK's, which is not always the Routecraft provider id: gemini reaches the SDK as google.

providerOptions wins over reasoning for the settings it names. The two merge per namespace and per setting inside it, so naming one setting replaces the mapped one and leaves the rest:

.enrich(llm('gemini:gemini-3-pro-preview', {
  reasoning: 'high',              // maps to thinkingConfig.thinkingLevel: 'high'
  providerOptions: {
    google: {
      thinkingConfig: { thinkingBudget: 2048 },  // sent instead
      cachedContent: 'my-cache',                 // sent as well
    },
  },
}))

Values nested deeper than a setting are replaced whole rather than deep-merged, so what you write for a setting is what is sent.

Two consequences worth stating outright:

• providerOptions wins over reasoning whichever level each came from. The two are folded together at dispatch, after context defaults and call-site options have merged, so a providerOptions set once in llmPlugin({ defaultOptions }) overrides the reasoning a specific call asks for, for the settings it names. If you want a house default that a call can still override, set reasoning in defaultOptions rather than providerOptions.
• Between the two levels, providerOptions is one value. Context defaults merge per option, so a call that sets providerOptions replaces the context's whole record rather than merging namespaces into it. The per-namespace, per-setting merge described above happens between reasoning and providerOptions, not between the two levels.

providerOptions is typed as raw records and the SDK is what validates it, so a namespace the SDK does not recognise is not an error: it is ignored, silently. That includes writing the Routecraft provider id where the SDK's namespace differs (gemini rather than google), which is the easiest way to get a setting that appears to be set and does nothing.

When the prompt does not fit

A prompt that exceeds the model's context window fails with AI1009 rather than the provider's own error, on both the buffered and the streaming path. It is the one dispatch failure no retry of the same input can fix, so it is marked non-retryable and the default .retry() predicate leaves it alone; a custom retryOn still decides for itself. Every other provider failure is rethrown untouched, with its retryability intact.

To test an error yourself, use isContextOverflow() from @routecraft/ai.