EMB-02 Released
Pico W voice terminal & control plane
A hand-wired push-to-talk voice assistant: custom PIO audio capture on the device, streaming speech recognition and permission-gated tools in a self-hosted backend.
Recruiter summary
- Problem
- The SPH0645 microphone's pin arrangement made the stock I2S receiver unusable on this build — and a voice terminal needs the whole chain (capture, transport, recognition, tools) to hold together.
- Contribution
- Hand-wired the device, wrote the PIO/DMA capture firmware and display code, and built the standalone self-hosted control plane any WebSocket client can join as a device.
- Result
- A released two-repository system verified end-to-end by a fully offline integration test in CI; in use it typically responds within a couple of seconds (observed, not benchmarked).
VOICE-SIGNAL
Interactive visualization · drag to inspect
3D ENGINEERING VIEW
The system as an interactive composition
Exploded visualization of microphone capture, PIO/DMA buffering, WebSocket transport, control plane, and display response.
Use the left and right arrow keys to rotate the scene. A complete text description is provided; no factual claim depends on the visualization alone.
The signal path — voice to display
An exploded layer composition: microphone above PIO capture, DMA buffer, WebSocket link, backend, and display.
-
Captures audio digitally over I2S. This microphone's pin arrangement made the stock I2S receiver unusable on this build.
-
A hand-assembled eight-instruction PIO program generates the bit and word clocks and samples the data line.
-
DMA streams samples into a static ring buffer — 16 kHz mono PCM ready to send without occupying the core.
-
One handshake at boot, then a long-lived channel: audio streams while you speak, so the transcript is final when the button is released.
-
FastAPI: device provisioning with show-once tokens, streaming STT, a permission-gated LLM tool loop, and secret masking.
-
The device draws the reply on screen. In use it typically responds within a couple of seconds — an observed figure, not a controlled benchmark.
Select a stage to inspect it
What it is
A Raspberry Pi Pico W wired by hand on a breadboard — 4″ ST7796 TFT, SPH0645 I2S microphone, push-to-talk button, RGB LED, buzzer — acting as a thin terminal: it captures audio, streams it, and draws replies. All intelligence lives in a self-hosted FastAPI control plane.
The part the stock library couldn’t do
The SPH0645’s pin arrangement made the standard I2S receiver unusable on this build. The fix is an eight-instruction, hand-assembled PIO program that generates the bit and word clocks and samples the data line, with DMA streaming samples into a static ring buffer. Audio leaves the device as 16 kHz mono PCM over a single long-lived TLS WebSocket — the handshake happens once at boot, and the transcript is already final when the button is released, because recognition streams while you speak. In use it typically responds within a couple of seconds — an observed figure, not a controlled benchmark.
The control plane
Published as its own product: any client speaking a 20-line WebSocket protocol is a device. It handles provisioning (auth tokens shown exactly once, rotate/revoke), presence, streaming STT, a provider-agnostic LLM loop with permission-gated tools (integrations ship disabled until explicitly authorized), runtime configuration with secret masking, history and logs, and a vanilla-JS admin console. A simulated-device example means the whole system runs and verifies with zero hardware.
Verification
The full pipeline — startup, login, settings, device provisioning, a complete voice turn over the WebSocket, persistence, presence, tool gating, logs — runs as one offline integration test with STT/LLM stubbed: no API keys, no network. The test also asserts the security properties (secrets never echoed back, device tokens never listed, gated tools inert without authorization, password change invalidates old sessions). CI runs it on every push. Firmware-side validation was manual, with documented bring-up procedures for the two riskiest pieces: the PIO receiver and TLS.
Evidence
Four photos of the device running, a wiring diagram, three admin-console screenshots, protocol documentation with sequence diagrams, and the passing CI run. Build close-up photos are on the capture list and will be added.
