Meta Open-Sources Muse Gadgets SDK for Wiring Its Muse Agent Into ESP32s and Raspberry Pis

TL;DR

Muse Gadgets is a new open-source project from Meta that lets you build your own hardware for Muse, the company's personal AI agent. The muse-gadget-sdk repo, published on October 2 under Apache 2.0, ships ESP32 firmware that already runs on 11 boards, a Linux SDK that turns a Raspberry Pi into a Muse-controlled machine, and 43 community "skills" for driving devices like Sonos speakers, Hue bridges and Roborock vacuums. Meta is also giving away 5,000 of its own Muse Home Link dongles to US subscribers. The catch: every gadget needs a Meta-issued SDK token, the whole thing is explicitly unsupported, and the Linux SDK gives the agent a shell.

What Meta actually shipped

The GitHub repo landed under Facebook Incubator on October 2, with three top-level pieces:

• ESP32 Device SDK. Firmware built on Espressif's ESP-IDF (pinned to v6.0.1) that connects a board to Muse over your Wi-Fi. Add a screen, a mic, a speaker or sensors and Muse can use them.
• Linux Device SDK. An installer for a Raspberry Pi 3B+, 4, 5 or Zero 2 W, or any Linux box with Bluetooth LE, that runs a musegadget service and exposes the machine to Muse.
• Community device skills. Markdown-only SKILL.md recipes that tell Muse how to talk to specific smart-home gear on your network.

Meta's own pitch, from the README: "It's built by hackers, for hackers, just for fun. Side effects of tinkering may include bricked boards, voided warranties, brownouts, or bankruptcies." Rare is the corporate README that lists bankruptcy as a known side effect, and rarer still the one where it reads as only half a joke.

Alongside the SDK, Nat Friedman, head of product at Meta Superintelligence Labs, announced the Muse Home Link, a USB-C-powered dongle that lets Muse reach smart devices on your home network. According to TechCrunch and Engadget, Meta built 5,000 of them and is giving them to Muse subscribers while supplies last. The Muse Gadgets site says it is free with an active Muse subscription in the US only, limited to one per subscriber, shipping in October.

How a gadget talks to Muse

The agent stays in Meta's cloud. Your board is a peripheral: it pairs through the Muse app on iOS or Android (Settings, Devices, with Developer mode switched on), shows up as MuseGadget-XXXXXX, and then holds a connection to Muse over Wi-Fi. On an ESP32 the status light walks you through it: orange for ready, blue to press the button and prove you are physically there, green once connected.

The interesting bit is the home-network tunnel. On boards with enough memory, the firmware lets Muse reach "the devices you already own and anything you build with a local HTTP API," per the ESP32 README. Boards without PSRAM, such as the classic ESP32 and the ESP32-C6, skip the tunnel; Muse can still control the board itself.

Think of it as giving a very capable remote assistant a walkie-talkie and a spare key. The assistant never moves into your house, but the walkie-talkie sits on your shelf, and with the key it can open whatever is on the LAN.

The boards that already work

The ESP32 README lists 11 supported boards. The quickest start is the ESP32-C5 DevKitC-1, which needs nothing but its built-in status light and BOOT button. Six boards run the full on-screen UI (an animated avatar, push-to-talk and settings), including the M5Stack StickS3 and StickC Plus2, the Seeed SenseCAP Watcher, the AIPI Lite and two Waveshare round AMOLED boards. Others, like the Home Assistant Voice Preview Edition and a 7.5-inch Seeed reTerminal e-paper, show status on an LED ring or a simple screen.

There is also a desktop simulator that runs the production UI and avatar renderer in a 412 x 412 window, so you can iterate on the screen without flashing anything. And a script, tools/muse/avatar.py, asks your Muse to redraw its own avatar as pixel art for your board's display, then builds and flashes it. Yes, the agent designs its own face for the hardware it is about to live in.

The agent-first build path

The recommended way to build is not to read the docs. It is to point a coding agent at them. Each SDK directory carries an AGENTS.md, and the README's quickstart installs Muse Code, Meta's coding agent, and has you type prompts like "Build this firmware for my ESP32-C5 DevKitC-1 and flash it" or "Add support for my board. It's an ESP32-S3 with 16 MB flash, a button on GPIO 0 and no PSRAM."

Meta says any agent that reads AGENTS.md works too. The practical wrinkle: flashing needs your USB serial port, so the quickstart runs Muse Code with --disable-sandbox. You still approve each command, but that flag deserves a second look before you paste it.

The manual path is standard ESP-IDF work: install v6.0.1, set the SDK token in idf.py menuconfig, build, flash, monitor. Builds are signed with an included development key and never turn on Secure Boot, so you can reflash as often as you like.

The Linux SDK gives Muse a shell

This is the part to read twice. Once installed, the Linux Device SDK exposes four commands to Muse:

• system.run: runs a shell command and returns output and exit code.
• file.read and file.write: 64 KB at a time.
• device.health: uptime, load, memory, disk and temperature.

Commands run as the account you installed for. The README is blunt: "If it can use sudo, so can Muse." The installer does ask before handing over the account and tells you whether it has sudo, which is the right default. Meta's example prompts include "Install Home Assistant on my Pi and tell me how to open it." Local scripts can also push messages to Muse with musegadget send-user-msg, no credentials needed, which makes a Pi a cheap bridge for any webhook or sensor you already have.

43 skills for the house you already own

The skill catalog counts 43 active skills: 42 device or device-family skills plus one shared Google Cast skill. They are plain Markdown. To use one, you give your Muse a link to the repo or paste the SKILL.md into chat. Meta labels them community-sourced and "not official integrations," and says they "may eventually be made available in the Muse product."

The coverage is broad: Sonos, Apple TV and HomePod mini, LG and Samsung TVs, Philips Hue, Lutron, Shelly and TP-Link Kasa plugs, Roborock and Roomba vacuums, Zigbee2MQTT, ESPHome, a ratgdo garage-door controller, and Moonraker-driven 3D printers. The skills are careful in a way you can copy. The Hue skill, for example, tells the agent to resolve resource IDs from fresh responses, only send fields the bulb supports, and confirm each light's resulting state rather than trusting a group acknowledgement. UniFi consoles and the camera skills are read-only by design.

If you write agent tools for a living, that catalog is a free reference for how to describe a local device API to a model: identify, prerequisites, workflow, verify.

The fine print

• Token-gated. Every gadget needs an SDK token from gadgets.muse.ai, "including ones you build for yourself," and the site says each token works on a limited number of devices. Open source firmware, closed pairing.
• Unsupported. The site calls the SDK "intended for personal tinkering" and "not a supported product or developer platform" that "can change, break, or stop functioning without warning." Do not build a business on it.
• Token in firmware. The token ships inside the binary, so Meta says to treat it as an identifier, not a password, and revoke it if it leaks.
• Pairing is not verified. Because these are community devices, the README says pairing "has no manufacturer verification and can't prevent an active man-in-the-middle attack." Pair on a network you trust.
• Encrypt the flash. Wi-Fi credentials and device tokens sit in NVS. Meta "strongly" recommends turning on NVS encryption, which is off unless you enable it in menuconfig.

The Hacker News thread split along predictable lines: tinkerers want the red-light, green-light desk lamp; privacy hawks want nothing from Meta on their LAN. Both camps are reading the same README correctly.

Why builders should care

Consumer agents have been trapped in phones and browser tabs. Meta just published the cheapest on-ramp yet for giving one a physical presence and a path into the home network, using parts you can buy for hobby money. Even if you never pair a board to Muse, the repo is a working reference design for agent peripherals: BLE pairing with a physical button confirm, a status-light state machine, a LAN tunnel gated on memory, and an AGENTS.md that lets a coding agent port the firmware to new boards.

It also tells you where the consumer-agent fight is heading. If the agent that owns the most surfaces in your house wins, Meta just outsourced the surfaces to hobbyists.

Key Takeaways

• Meta published muse-gadget-sdk on October 2 under Apache 2.0: ESP32 firmware, a Linux SDK and a skills catalog for its Muse agent.
• The ESP32 firmware supports 11 boards today, six with a full on-screen UI, and boards with PSRAM get a tunnel that lets Muse reach devices on your LAN.
• The Linux SDK gives Muse system.run and file access with your account's permissions, including sudo if the account has it.
• 43 community skills cover speakers, TVs, lights, plugs, vacuums, gateways and printers; network and camera skills are read-only.
• Meta built 5,000 Muse Home Link dongles, free to US subscribers, one each, shipping in October.
• Pairing requires a Meta SDK token, is not manufacturer-verified, and the whole platform is explicitly unsupported.

Sources: facebookincubator/muse-gadget-sdk on GitHub, ESP32 Device SDK README, Linux Device SDK README, Skill catalog, Muse Gadgets, TechCrunch, Engadget, Hacker News discussion