Firmware
The esp-agents-firmware repository (https://github.com/espressif/esp-agents-firmware) is the device-side SDK and reference examples for ESP devices that talk to the ESP Private Agents platform.
If you've bought a pre-flashed board, you don't need this section — you can pair it with ESP RainMaker Home and start talking. This section is for makers and integrators who want to flash, customize, or build on top of the firmware.
What's in the repo
Two example applications:
| Example | Default agent | What it does |
|---|---|---|
| Voice Chat | friend | Generic voice-chat companion device. Talks to the Friend agent (or your own custom agent). Can also control RainMaker smart-home devices through a remote MCP server. |
| Matter Controller | matter_controller | Voice-chat plus a Matter Controller (and optional Thread Border Router) — say things like "turn off the kitchen light" and have it execute as a Matter command. |
Both are built on ESP-IDF and run on ESP32-S3 boards.
How it talks to the platform
- Wake word: on-device, "Hi, ESP" (also tap-to-wake on the screen).
- Audio defaults: 8 kHz PCM up to the cloud, 16 kHz PCM down. Both rates are Kconfig settings — change them at build time if needed. AEC is available as an opt-in.
- Auth: long-lived refresh token compiled into the device.
- Local tools: a small set of device-resident functions the cloud agent can call. See Local Tools.
Three ways to get started
| Path | Time | What you need |
|---|---|---|
| Quick flash (pre-built images via ESP Launchpad) | 5 min | A supported board, a USB cable, Chrome / Edge. |
| Build from source | 30 min first time | A supported board, ESP-IDF v5.5.2, a working terminal. |
| Customize / extend | hours+ | Same as build-from-source plus C familiarity. |
Read on:
- Supported Boards
- Quick Flash (ESP Launchpad)
- Build from Source
- Voice Chat Example
- Matter Controller Example
- Local Tools
- Audio & Wake Word
- Pointing at a Private Deployment
License & status
The firmware is open-source (Apache-2.0; license headers in source files). The repo is on GitHub at https://github.com/espressif/esp-agents-firmware. Contributions are welcome — see the repo's CONTRIBUTING for the patch process.