LLM (Model)
The LLM (Large Language Model) is the brain of your agent. Each agent has two model slots:
- Text Model — the primary model for chat. Required.
- Speech Model — the voice model for real-time voice conversations. Optional. Defaults to "No speech model selected" for text-only agents.
Both are set in the Basic Info section of the agent editor.
Picking a model
Each model slot has a selector button showing the currently chosen model. Click it to open the model dropdown.
The dropdown provides:
- A search box ("Search text models…") at the top — type to filter by name, provider, or description.
- Models grouped by provider (Anthropic, Amazon Bedrock, Google, OpenAI, etc.).
- A provider header note: "API key not set" — when a provider's credentials haven't been configured. Those models appear greyed out and can't be used until a super-admin adds the key (see Deployment Settings → LLM Provider API Keys).
- A model details button (info icon) on each row — click it (not hover) to see pricing per million tokens and capabilities.
The Compare button
Both Text Model and Speech Model headings have a Compare button. Click it to open a side-by-side comparison table of available models — useful when deciding between options on cost, speed, or capabilities.
Supported providers
The model catalog includes models from multiple providers. Whether a model is available depends on whether it uses AWS Bedrock (no API key needed, uses IAM) or a direct API key:
| Provider | Access method |
|---|---|
| Amazon Bedrock (Nova Lite, Nova Pro, etc.) | IAM — available by default on Bedrock-enabled deployments |
| Anthropic (Claude Sonnet 4.6, Opus 4.7, Haiku 4.5, etc.) | Direct API key or Bedrock-hosted |
| Google (Gemini 3.1 Pro, Gemini 3 Flash, etc.) | Google AI Studio API key |
| OpenAI (GPT-5.5, o3, o4-mini, etc.) | OpenAI API key |
| DeepSeek (V3.2, V4 Pro, V4 Flash, etc.) | Bedrock-hosted or direct API key |
| Moonshot Kimi (K2.5, K2 Thinking, etc.) | Bedrock-hosted or direct API key |
| Alibaba Qwen (Qwen3 Max, Qwen-VL Max, etc.) | Alibaba DashScope API key |
| xAI (Grok 4.20, Grok 4.1 Fast, etc.) | xAI API key |
| Z.AI (GLM-5, GLM-4.7, etc.) | Bedrock-hosted |
| MiniMax (M2.5) | Bedrock-hosted |
| Qwen3 (80B, 30B, Coder Next, etc.) | Bedrock-hosted |
If a provider row is greyed out with "API key not set", ask your super-admin to add the key on the Deployment Settings → LLM Provider API Keys page.
Capability matching
Each model has a capability profile. The dropdown flags mismatches visually:
| Visual state | Meaning |
|---|---|
| Normal | Model is available and compatible. |
| Greyed out | Provider API key not configured. |
| Red | Model is missing a capability your agent needs (tools, vision, thinking). |
| Capability | When you need it |
|---|---|
| Tools | The agent has any tool attached. |
| Vision | The agent processes images or multimodal input. |
| Thinking | Extended reasoning mode is enabled. |
For a model shown in red, either pick a different model or remove the conflicting requirement.
Thinking budget
Some models (primarily Claude) support extended thinking — the model reasons internally before replying. This is configured via a four-option picker that appears in Basic Info when a thinking-capable model is selected. If the selected model doesn't support thinking, this picker is hidden.
| Preset | Tokens reserved | When to use |
|---|---|---|
| Off | 0 | Default. Use for most agents. |
| Low | 2,048 | Light extra reasoning. |
| Medium | 4,096 | Multi-step questions. |
| High | 8,192 | Hard problems requiring careful reasoning. |
Higher budgets mean deeper reasoning but longer response times and higher cost. Start with Off and only increase if the agent gives wrong answers on complex questions.
Speech models
Speech models handle real-time voice conversations. Today the catalog ships two speech options, both on Amazon Bedrock:
- Amazon Nova Sonic
- Amazon Nova 2 Sonic
Select "No speech model selected" (the default) for text-only agents.
Speech model performance is independent of the text model — changing one does not affect the other.
Streaming
Agents stream their replies by default — text appears word by word as the model generates it. This is on by default and is not currently surfaced as a toggle in the editor. Contact your platform admin if you need to disable it for a specific agent.
Switching models on a live agent
You can change an agent's model at any time. Existing conversations keep the model that was active when they started; new conversations pick up your updated choice.
Tips
- Start with a cheaper model. Most agents work fine on a fast, affordable model. Only upgrade if you see a quality problem.
- Use the Compare button. Before committing to a model, open Compare to see pricing and capabilities side by side.
- Don't pay for capabilities you don't use. Don't pick a vision model when the agent never processes images.
- Use thinking sparingly. Off is right for most agents. Turn it on only for agents that fail on multi-step reasoning tasks.
- Watch the bill. Switching from a small to a frontier model can multiply cost 10× or more. Check Analytics regularly.