Voice Chat Example
examples/voice_chat/ in the firmware repo. The default agent is friend — a friendly voice companion that can chat and (through MCP) control your RainMaker smart-home devices.
What it does
A user wakes the device with "Hi, ESP" (or a tap), speaks a question or command, and hears the agent's spoken reply. The device emotes — its on-screen face changes based on conversation tone — and supports a few local tools for controlling itself.
Local tools
The Voice Chat firmware exposes these tools to the agent:
| Tool | What it does |
|---|---|
set_volume | Adjust speaker volume (0–100). |
set_emotion | Set the on-screen emoji from a fixed set: Sad, Happy, Confused, Shocked, Angry, Cry, Wink, Neutral. |
set_reminder | Schedule a local reminder that fires later. |
get_local_time | Returns the device's current time. |
These run on the device, not in the cloud. The agent decides when to call them; the firmware executes; the result goes back to the agent. See Local Tools.
RainMaker integration (out of the box)
The Friend agent's configuration includes a remote MCP tool pointing at https://mcp.rainmaker.espressif.com/api/mcp. This means a Friend device can also control any RainMaker smart-home devices on the same Espressif account — without any firmware-side changes.
This is convenient but it means the device doesn't talk to RainMaker directly; the agent does, with your OAuth-authorized RainMaker account.
Default agent vs. your own agent
The firmware ships with friend baked in (CONFIG_AGENT_SETUP_DEFAULT_AGENT_ID="friend" in sdkconfig.defaults). You don't have to change the firmware to use a different agent — you can:
- Build a custom agent in the Admin Dashboard.
- Share the agent and scan the QR with your phone — see Assign an Agent.
The device picks up the new agent ID on the fly.
If you want to change the default (compiled in), edit CONFIG_AGENT_SETUP_DEFAULT_AGENT_ID via idf.py menuconfig → ESP Agent Setup Configuration, then rebuild.
On-device behavior
| Action | What happens |
|---|---|
| Power on | Self-test, then either pairing QR (unconfigured) or normal startup with the configured agent. |
| "Hi, ESP" | Chime; mic opens; you speak; mic closes when you stop. |
| Tap screen | Same as wake word. |
| Tap during agent reply | Interrupts the agent (barge-in). |
| 15 seconds of silence | Goes back to sleep. |
| Tap-and-hold top of screen for 10 s | Factory reset (board permitting). |
Boards supported
All four supported boards run Voice Chat. See Supported Boards.
Common customizations
- Add a local tool: see Local Tools → Adding a tool.
- Change the wake word: WakeNet is on-device; swap to a different model in
idf.py menuconfig→ WakeNet. - Disable the screen (display-less devices): comment out
app_display_*calls inmain/app_main.c. Hand-editing required; the firmware doesn't have a Kconfig switch for this yet. - Adjust audio sample rates: menuconfig → App Common Config. Default is 8 kHz upload / 16 kHz download.
Where to look in the code
| Path | What's there |
|---|---|
examples/voice_chat/main/app_main.c | Entry point, init order. |
examples/voice_chat/main/app_tools.c | Tool registration (app_tools_register). |
examples/voice_chat/main/app_emotion.c | Emotion → on-screen face mapping. |
examples/voice_chat/agent_config.json | The Friend agent's definition (system prompt, tools, model). |
components/app_common/ | Shared audio, LVGL, networking. |
Read next
- Local Tools — add or modify tools.
- Audio & Wake Word — sample rates, AEC, wake-word models.
- Pointing at a Private Deployment — switch from Espressif's hosted platform to your own.