AI Agent
ROX
A J.A.R.V.I.S.-style voice assistant for Windows: it wakes on two claps or the word "ROX", understands the spoken request and acts on the computer — often before you finish the sentence.
- fast-path response time
- 15 ms
- tools on the computer
- 63
Architecture
- Python
- Electron
- Kokoro TTS
Features
- Wakes on a double clap with an adaptive floor or on the word "ROX", detected locally with Silero VAD and Whisper — no API key needed to listen.
- Acts while you speak: partial transcription runs every second, and a complete command fires before the sentence ends.
- A fast path that learns: phrases resolved by the model become local shortcuts (~15 ms) and never go through the LLM again; destructive actions are kept out of it.
- 63 tools on the computer: apps and windows, per-app volume, media, mouse and keyboard, clicking buttons by name, seeing and reading the screen, files, clipboard, timers, reminders and weather.
- Routines by time or by battery ("below 20%") with a tolerance window; reminders fire even when late.
- Persistent profile — apps used, nicknames, transcription corrections — and a searchable conversation diary; every destructive action requires explicit confirmation.
- Speech policy "actions are silent, questions are spoken": the assistant only speaks when it needs you.
Infrastructure
- Python 3.12 backend (threads + asyncio) handling audio, speech, AI, routines and reminders; an Electron + Three.js overlay that never steals focus, connected over WebSocket.
- Transcription with faster-whisper on GPU (CUDA) with CPU fallback; voice detection with Silero ONNX.
- Kokoro neural voice on the GPU, with Edge TTS and pyttsx3 as fallbacks — the last one works offline.
- Reasoning by Claude (Haiku by default, Sonnet for long requests) through the CLI on the plan's quota; tools are served by a local MCP server. PowerShell installer with auto-start and a live usage dashboard.
What it solves
- Voice commands with no keyboard, with near-instant replies for routine requests — and more and more becomes routine.
- Cuts repetitive daily tasks — opening programs, adjusting volume, remembering appointments — down to a sentence.
- Listens and transcribes locally: audio never leaves the computer; only the understood request goes to the model.
- Gets better over time: transcription corrections and nicknames stay in memory.
Want something similar?
Tell me how your process works today. In a quick call I get to know the problem and send a proposal with scope, timeline and price.