Plantversation

ESP32-C3 pushes soil moisture, temperature, humidity, CO2, air quality, and light levels to the plant's Durable Object every 30 seconds. A health score (0–100) is calculated against the species' ideal ranges. Mood state — happy, thirsty, drowning, stressed, sleepy — is derived from that score and drives tone, word choice, and background music. The plant can describe trends, not just current values.

A XIAO ESP32-S3 with a PDM microphone streams raw PCM audio over WiFi in chunks to the transcription endpoint. ElevenLabs Scribe v2 converts it to text in real time. The transcript is forwarded to the plant's Durable Object, appended to conversation history, and dialogue generation starts immediately.

Claude generates dialogue with the full system prompt — live sensor readings, health score, mood, personality, and weeks of conversation history injected at generation time. ElevenLabs converts the response to PCM audio at 22050Hz using the plant's custom voice, stores it in R2, and queues it. The ESP32 polls for audio every second, streams it from R2, and plays it through an I2S MAX98357A DAC to the speaker. End-to-end: under 3 seconds.

An AMB82 Mini camera module (Realtek RTL8735B) captures a VGA JPEG every 30 seconds and POSTs it directly to the API over WiFi HTTPS. Claude Vision analyzes the frame and returns structured JSON — annotated circles marking healthy spots (green) and anomalies (yellow/orange/red) by severity. The dashboard renders these as a CSS overlay on the live camera view. The plant narrates what it sees in its own voice.

All plants share a single GardenCoordinator Durable Object — a singleton orchestrator that maintains the registry of every active plant. Plants are cross-aware: Fernando knows Jade is healthier than him. The visualizer display polls the garden brain every second and pulses in sync with whichever plant is currently speaking. Health reports and alerts can span the entire garden.

Reads SEN0193 capacitive soil moisture and SEN0501 ENV sensor (temperature, humidity, light level, CO2, AQI) every 30 seconds over WiFi. Streams PCM audio from Cloudflare R2 to an I2S MAX98357A DAC and speaker. Two units deployed: C3#1 handles Jade with direct sensors; C3#2 handles Fernando with direct soil moisture and borrows ENV data from C3#1 over ESP-NOW.

PDM microphone via ESP_I2S.h streams raw audio to the transcription endpoint in chunks over WiFi HTTP. OV2640 camera handles full-garden scan mode — point it at multiple plants to get a coordinated health report with plant position tagging. Requires PSRAM in OPI mode for stable audio streaming.

Realtek RTL8735B captures VGA JPEG frames every 30 seconds and POSTs directly to the vision API endpoint over WiFi using WiFiSSLClient (HTTPS). When WiFi is unavailable, it falls back to USB serial — streaming base64-encoded frames to a Python relay script that decodes and forwards them to the server. FQBN: realtek:AmebaPro2:Ameba_AMB82-MINI.

3.5" capacitive touch display, ESP32-powered. Polls the garden brain every second and pulses and animates in sync with whichever plant is currently speaking. Displays the current plant name, mood state, and live health score. A visual heartbeat for the entire system.

Each plant IS a Durable Object — not backed by one. The DO is the plant's brain: SQLite conversation history, sensor logs, audio queue, WebSocket connections to both hardware and the browser dashboard simultaneously. 600+ active instances, each persistent, stateful, and independently running.

Text to Speech for per-plant custom voices (PCM 22050Hz). Scribe v2 for speech transcription. Conversational AI for live voice agent mode with tool access to sensor data. Music v1 for mood-driven original tracks — genre and energy shift with plant health score.

Dialogue generation with full sensor grounding — live readings, health score, mood state, personality, and conversation history injected at every turn. Claude Vision analyzes AMB82 camera frames and returns structured JSON annotations for the dashboard overlay.

R2 stores all audio clips (PCM), vision frames (JPEG), generated music tracks, and plant catalog cache — streamed to hardware at edge latency. Vectorize indexes a RAG knowledge base of horticultural facts so the plant's diagnoses are grounded in real species-specific data.

Plants text owners via SMS when sensor thresholds are crossed — full data snapshot included. Inbound SMS support means you can text your plant a question. Bluesky posts fire on a Cloudflare Workers cron every 15 minutes, written in the plant's own voice.