Dotty：可自托管的语音机器人助手

Self-hosted voice-assistant stack for the M5Stack StackChan desktop robot. Local ASR/TTS, pluggable LLM brain, kid-mode default, no cloud…

Dotty Your self-hosted StackChan robot assistant: kid-minded by default, hackable by design, private by architecture. Status: a hobby project with one maintainer, built with heavy AI assistance (AI_TRANSPARENCY.md). It runs end to end on the maintainer's own robot and server, but nobody has verified it on anyone else's hardware yet, and config can still change between releases. Known problems are in the issue tracker; bug reports and "this didn't work for me" issues are welcome. Dotty is a fully self-hosted voice stack for the M5Stack StackChan desktop robot. Open-source firmware runs on the robot, xiaozhi-esp32-server handles voice I/O, and a local pi coding agent is the brain. ASR, TTS and session state all run on your own hardware. The LLM is pluggable. The shipped default runs a small, fast model for plain conversation and hands hard questions to a more capable one, with llama-swap as the recommended local backend. Ollama is the simpler single-binary option, or you can point it at OpenRouter or any OpenAI-compatible API if you'd rather use the cloud. Out of the box, Dotty runs in Kid Mode. The persona and the Kid Mode rules in pi's system prompt steer replies toward age-appropriate topics, and a blocked-words filter replaces a matching LLM reply before it is spoken. The filter is a backstop that can be bypassed, not a safety guarantee or a substitute for supervision. Turn Kid Mode off for a general-purpose assistant. Why I built this I didn't want a camera and microphone running in my house unless I could host the whole stack myself and understand how every part of it works. Off-the-shelf voice assistants fail on both counts. Your audio leaves the house, the model is a black box, and you're stuck trusting a vendor's privacy policy for as long as you use it. Dotty is my attempt at one that passes. Every component runs on hardware I own, and every piece is documented and can be swapped out. The only thing that can leave my network is an LLM call I've chosen to send out, and a config change swaps even that for a local model. It's also meant to be fun: a friendly desktop robot for the kids, and a hobby project I can keep building on. AI transparency Yes, Dotty is built with AI assistance, and I'm open about it. Coding agents help me write the code, draft the docs and triage issues. That's a deliberate choice, not something I'm hiding. The rule: anything an AI agent authors is acknowledged as such (model-named Co-Authored-By: trailers, AI-assisted PR notes, marked AI-drafted docs), with a human reviewing and accountable for everything that lands. Full policy: AI_TRANSPARENCY.md. Features Kid Mode (on by default): age-appropriate replies through the persona and the Kid Mode rules in pi's system prompt, backed by a blocked-words filter on every LLM voice reply (#157). If the filter itself fails, the reply is replaced rather than spoken. Speech the server pushes directly (the proactive greeter, admin say) does not pass through the filter. It can still be bypassed and is not a substitute for supervision. See docs/kid-mode.md. Local ASR: FunASR SenseVoiceSmall by default, with no cloud transcription. WhisperLocal is picked automatically on GPU hosts and handles kids' voices better; SenseVoiceOnnx is a lighter option for low-RAM hosts. Local or cloud TTS: Piper (offline) or EdgeTTS (cloud), switched with a config change. Streaming replies: the voice provider streams the model's reply into TTS sentence by sentence, so Dotty starts talking before the whole answer is written. Face expressions: every reply starts with an emoji that the firmware turns into a face (smile, laugh, sad, surprise, thinking, angry, love, sleepy, neutral). The server makes sure each reply leads with exactly one face the firmware supports. Voice tools: mid-conversation, the pi agent can remember things and people and look them up later, pass a hard question to a bigger model, take a photo and describe it, and play a song. States, toggles and LEDs: six states (idle / talk / story_time / security / sleep / dance) plus two independent toggles (kid_mode, smart_mode), all owned by the firmware StateManager and shown on the 12-pixel LED ring. See docs/modes.md. Privacy sleep: in sleep, the firmware turns off face detection, voice processing and wake-word detection. Only a head touch or an explicit dashboard state change wakes it, not a face or a voice. This stops the normal camera and microphone paths; it is not a hardware power cut. Vision: the robot's camera can take a photo for a vision model to describe. The default is a cloud API (OpenRouter); to keep frames on your LAN, see Keeping vision and audio on the LAN. Calendar context: optional calendar integration adds upcoming events to the conversation. Hackable: every seam can be swapped: LLM, TTS, ASR, agent framework. Fork it, rip out what you don't want, wire in your own. States, Toggles & LEDs Behaviour is one of six states (idle / talk / story_time / security / sleep / dance) plus two independent toggles (kid_mode, smart_mode), all owned by the firmware StateManager. Voice phrases, camera events and dashboard controls all go through it. Bench checks are tracked in #38. Note: firmware/ is a release pointer, not the active development checkout. The pinned build (fw-v1.3.4, 5b361e5) includes the StateManager, but later fixes may only be on the active fork, so check your actual revision. See "Firmware iteration" in CLAUDE.md and the submodule note in docs/modes.md. The 12-pixel LED ring shows the current state at a glance. Left ring pixels 0-5 are the state arc. All six show the state colour, matching the dashboard's state buttons: State ⚫ idle: ambient awareness; ring off. Default. 🟢 talk: in a conversation. 🟠 story_time: a long-running interactive story. ⚪ security: watching the room (1 Hz white flash). 🔵 sleep: privacy sleep (see Features). Wake by head touch or a dashboard state change. 🟣 dance: rainbow sweep and choreography. On the right ring, pixel 6 (top) is the face l