Glossary
Definitions for terms used throughout these docs. If you see a term in a page that isn't clear, it's probably defined here.
Agent
An AI assistant that an admin builds on the Admin Dashboard. Has a name, an LLM, a system prompt, a list of tools, optionally one or more knowledge bases, and (per end user) any connectors the agent's tools require. End users chat with agents; agents call tools, look things up in knowledge bases, use the user's connectors, and reply.
Admin
A user who can build and manage agents on the Admin Dashboard. Distinct from an end user. One platform can have many admins; one of them is the super-admin.
Agent Template
A pre-configured agent definition that admins can use as a starting point. Read-only in the dashboard today — Espressif (or your platform team) seeds the template catalog.
Barge-in
In voice mode, when the user starts speaking while the agent is still speaking, the platform's voice model detects the overlap, cancels the assistant turn, and tells the client over the WebSocket via a barge_in envelope. The client uses that signal to stop local playback.
Bedrock
Amazon Bedrock — AWS's hosted LLM service. The platform uses Bedrock for many models (Claude, Amazon Nova, Titan, Llama, Mistral, DeepSeek, Qwen, Kimi) and for voice (Nova Sonic). Account-level access has to be requested before models will work; third-party Bedrock models also require an AWS Marketplace subscription.
Cognito
AWS's identity service. The platform uses two Cognito user pools by default — one for admins, one for end users. You can swap them for unified or external pools.
Connector
A user's authorization for an agent to act on their behalf in a third-party service (Google Drive, Slack, Notion, etc.). Implemented as OAuth — the user authorizes once, the platform stores an encrypted token, and agents use it as needed.
→ End User → Connectors, Admin → Tools
Conversation
A back-and-forth thread between an end user and one agent. Each message is stored. A conversation can be text, voice, or mixed.
Default Agent
The agent ID baked into a device's firmware (friend for Voice Chat, matter_controller for Matter Controller). The device talks to this agent until it's reassigned via QR scan.
End User
Someone who chats with agents — typically via a paired device or via the User Dashboard. Doesn't build agents themselves.
Friend
The default agent that ships with the Voice Chat firmware example. A friendly voice companion that can also control RainMaker smart-home devices.
Knowledge Base (KB)
A collection of documents an agent can search during a conversation. Files (PDF, DOCX, etc.) and web URLs are ingested, chunked, embedded, and stored. The agent retrieves relevant chunks at chat time.
Local Tool
A tool that runs on the device (firmware) rather than in the cloud. Examples: set_volume, set_emotion, get_local_time, control_device (Matter). The agent sends a tool_request over the WebSocket; the firmware executes; the result is sent back.
→ Firmware → Local Tools, WebSocket Protocol → Tool Calls
LLM
Large Language Model. The brain of the agent. Each agent has one text model (for chat) and optionally one speech model (for voice). The platform's provider-agnostic LLM layer supports models from AWS Bedrock plus Anthropic, OpenAI, Google, xAI, DeepSeek, Kimi, and Qwen — eight registered providers in total.
Matter
The smart-home interoperability standard. The Matter Controller firmware example lets a device commission and control Matter accessories (lights, plugs, dimmers) via voice.
→ Firmware → Matter Controller
MCP (Model Context Protocol)
An open protocol for exposing tools to LLM agents. Remote MCP servers can be added to any agent — the platform discovers their tools, optionally authorizes via OAuth, and the agent can call them.
→ Admin → Tools, Admin → Add an MCP Tool
Platform / Gateway
The cloud backend — a FastAPI service running as one or more auto-scaled tasks on AWS ECS Fargate, behind an Application Load Balancer. Handles authentication, LLM routing, tool execution, audio streaming, and persistence. Sometimes called "the gateway" because it's what every client talks to.
RainMaker
Espressif's existing IoT platform — rainmaker.espressif.com. ESP Private Agents reuses RainMaker's account system for sign-in, and the Friend agent can control RainMaker-managed smart-home devices via MCP.
Refresh Token
A long-lived token a user can copy from their User Dashboard profile. Used to authenticate ESP devices and other non-browser clients to the gateway.
Remote (MCP) Tool
A tool implemented by a remote MCP server. There are two backend types:
remote_fetch— a discovered MCP server: the platform fetches its tool list, optionally authorizes via OAuth, and lets the agent pick from the discovered tools.remote— a single hand-defined remote tool with a fixed schema, served by an HTTP endpoint.
Provider Credentials
The set of API keys the deployment holds for non-Bedrock LLM providers (Anthropic, OpenAI, Google, xAI, DeepSeek, Kimi, Qwen). One key per provider per deployment. Stored encrypted in DynamoDB and used by every agent that picks a model from that provider.
→ Deployment Settings → LLM Provider API Keys
Quota
A user's daily usage allowance, measured in the units the platform admin set up (typically a weighted combination of input + output tokens, plus audio seconds for voice). Enforced server-side over a 24-hour rolling window; surfaced in the User Dashboard's Usage & Quota page.
Personalization
Per-user standing instructions (display name, preferences) that the platform attaches to every chat. Distinct from agent-level system prompts and per-conversation memory.
Streaming
Token-by-token delivery of an assistant reply over the WebSocket. The gateway emits assistant_delta envelopes during streaming and a final assistant envelope when complete. On by default for new agents; enabled per agent.
Voice Chat
The firmware example, the User-Dashboard mode, and the gateway voice path that together let an end user speak with an agent. Powered by Amazon Nova Sonic on the gateway side.
→ End User → Voice Chat, Firmware → Voice Chat
Matter Controller
The firmware example for using a device as a Matter accessory controller — the device commissions accessories itself rather than relying on a separate hub. Voice commands invoke local Matter tools that act on the controlled accessories.
→ Firmware → Matter Controller
ESP Boards
The four boards the firmware ships support for: EchoEar Core Board v1.2 (Espressif's primary voice-agents reference board), ESP-BOX-3, M5Stack CoreS3, and M5Stack CoreS3 + H2 gateway. See the firmware section for the matrix of which examples support which board.
Sharing
The flow for handing an agent to an end user — the Admin Dashboard generates a share URL and a QR code. End users open the URL (or scan the QR with the mobile app) to start using the agent.
Speech Model
The LLM used for voice — typically Amazon Nova Sonic. Distinct from the text model. Optional; agents without one are text-only.
Super-admin
The deployment-level identity set at install time via the SuperAdminEmails CloudFormation parameter. Has authority over deployment-management actions (changing Cognito pools, updating versions). Distinct from regular admins.
System Prompt
The standing instructions an admin gives an agent. The model reads it before every reply. Sometimes called "Instructions" in the Admin Dashboard.
Thinking Budget
A token budget reserved for extended reasoning before the model replies. Presets: Off / Low (2,048) / Medium (4,096) / High (8,192). Only relevant for models that support extended thinking (like Claude).
→ Admin → LLM → Thinking budget
Tool
Anything the agent can call beyond raw text generation. The platform supports five tool types:
local— a tool implemented on a connected client (the firmware on a device, or a local MCP server). The agent sends atool_requestover the WebSocket; the client executes and replies.remote— a single hand-defined remote tool: name, description, input schema, and an HTTP endpoint.remote_fetch— a discovered MCP server: tool list is fetched, the admin picks which tools the agent can use.cloud— a server-side tool implemented by the gateway itself (e.g. web search wrappers). Distinct from a Knowledge Base.device— a device-resident tool reached through RainMaker (used by the Friend agent's smart-home flows).
Knowledge Bases are a separate object — they're attached to an agent like tools, but they're not classified as tools.
Tool Template
A pre-configured tool definition admins can stamp into any agent. Read-only in the dashboard — seeded by Espressif or your platform team.
Wake Word
The phrase that switches the device from sleep to listening. Default: "Hi, ESP". Detected on-device by Espressif's WakeNet engine using a WN9-series model. Tap-to-wake on the screen also works.
→ Firmware → Audio & Wake Word
WebSocket Protocol
The single bidirectional protocol every client (device, browser, custom integration) uses to chat with the gateway. Carries text, voice (binary frames), tool calls, and lifecycle events.