06 — DELIVERED BUILDS
Completed Systems
Custom AI-integrated systems and automation builds designed, built, and delivered for clients. Each system is documented without identifying business or personal detail.
Custom AI-integrated systems and automation builds designed, built, and delivered for clients. Each system is documented without identifying business or personal detail.
Interacting with AI tools through terminal windows and text interfaces is impersonal, interruptive, and requires technical fluency every time. The goal was to create a persistent, human-like presence on the operator's desktop — an always-on-screen animated character that listens to natural spoken voice, processes requests through a connected AI brain, and replies in a natural synthesised voice, all without the operator needing to open a terminal or type a single word.
The system runs as two parallel layers. The voice loop (Python) captures microphone input continuously using the system's default audio device. Each short recording is checked for audio level — if the input is below threshold, the loop skips silently and records again. When speech is detected, it is transcribed locally on-device using a Whisper speech recognition model running on CPU with voice activity detection filtering, so background noise does not produce phantom responses.
The transcript is forwarded over SSH to a remote AI brain machine, which processes the request and returns a natural-language reply. That reply is immediately sent to ElevenLabs for neural text-to-speech synthesis using a custom voice profile, and the resulting audio is played through the system speakers. The full cycle — from spoken word to spoken reply — is continuous and hands-free. Saying "stop", "exit", or "quit" cleanly shuts down the loop.
The visual layer runs as a native Electron desktop application. The window is frameless and fully transparent, floating above all other applications without an OS border or taskbar presence. Inside the window, Candy is rendered as a fully animated SVG character — her pupils follow the cursor in real time using smooth interpolation, her eyelids blink at natural intervals, her hair sways on idle, and her facial expression shifts when she is speaking. A text chat panel is embedded alongside the avatar for typed interaction when voice is not available.
A separate full-body presence layer (PySide6) renders a PNG cutout of Candy anchored to the bottom-right corner of the screen. She gently bobs with a sinusoidal floating animation, stays on top of all windows at all times, and can be repositioned anywhere on screen by dragging. The visual presence persists independently of whether the voice loop is active.
The business was operating order intake manually — phone calls, handwritten notes, and manual entry into a legacy portal — across multiple depot locations. The goal was to digitise the customer-facing order process, introduce AI-assisted processing, and build a clear approval workflow that kept humans in control at every step while reducing admin overhead.
Customers access a mobile-first web portal from any device. They browse a product catalogue organised by category and SKU, select products and quantities, specify delivery method, preferred depot location, and delivery date, then submit the order as a request. No pricing is visible to customers except on invoices.
When an order is submitted, an AI processing layer immediately validates the payload — checking required fields, classifying the order as simple or complex, cross-referencing available stock and transport data, and generating a plain-language staff summary. The AI does not create or modify anything in the live business system at this stage.
The responsible operator receives an email notification containing the AI-generated order summary and a one-click approval link. Until that link is clicked, the order remains a draft. Nothing enters the operations portal without explicit human approval. Once approved, a structured order bundle is prepared for entry into the existing business management portal, with all required fields pre-mapped and a document checklist to ensure pricing does not appear on non-invoice documents.
Once the order is confirmed in the portal and a reference number is generated, the system links the customer submission to the portal order, prepares a customer confirmation, and tracks the order through status stages — New, Allocated, Delivered — with freight docket generation tied to each stage.
Managing multiple AI agents, local services, and distributed machines from separate tools — command-line terminals, browser dashboards, text config files — was fragmented, slow, and required technical knowledge to operate. The goal was to consolidate all of that control into one installed desktop application: a single surface where an operator can start and stop agent services, monitor their live state, check network machines, track performance metrics, and launch connected web portals without touching a terminal.
The application installs directly on the operator's workstation as a native desktop app and can be launched from the system tray. At startup it loads persisted configuration from the local system, establishes a connection to the AI gateway service, and begins polling all configured services and network machines for live status. All settings — machine addresses, service ports, API endpoints, linked web apps — are written to a local configuration file and reloaded across sessions.
The operator console is organised into tabbed panels. The Agents panel shows the Hermes AI gateway and processing engine in real time — running or stopped, active session count, background worker count, and queued task totals pulled live from the gateway's kanban board. The Machines panel pings each configured network machine and reports online or offline status with one-click SSH access. The Web Apps panel surfaces all linked portals and dashboards as single-click launchers.
The Electron main process acts as the privileged control layer, able to spawn the AI gateway process directly and monitor it with automatic restart logic — up to five recovery attempts with exponential backoff before alerting the operator. An embedded Express server handles REST operations and proxies all renderer requests to the gateway API through a strict endpoint allowlist, keeping the renderer isolated from direct network or shell access. A Python bridge script enables bidirectional config sync and status reporting between the desktop application and the AI agent system, allowing cross-control from the agent terminal as well.
Structured events from every component are streamed into a live event log panel, giving the operator a real-time audit trail without external monitoring infrastructure. Revenue and usage metrics are pulled from a configurable external API endpoint and surfaced in the Profit panel. A local login store tracks credentials and links for all connected services, accessible with a single click.