UtterX: Closed-Loop Agentic Execution Runtime

System Overview

UtterX is a closed-loop agentic execution runtime that enables LLMs to reliably interact with complex web applications and live browser DOMs. Rather than relying on open-loop generation (which degrades rapidly on dynamic SPAs), UtterX implements a programmatic verifier and a Reflexion-style self-healing correction loop.

Key Architectural Innovations

• Closed-Loop Self-Healing Runtime: An LLM generates executable JavaScript from natural language intent, injects it into a live DOM, and evaluates execution success using a visual diff gate and console error taxonomy. Runtime failures are routed back into targeted fixers.
• Unlabeled Reward Signal Pipeline: Engineered a multi-tier reward pipeline without manual labels:
  • Structural signals: Selector rank stability and turn efficiency.
  • Outcome signals: DOM mutation diffs and intercepted network event logs.
  • Behavioral signals: User reset rate calibrated as implicit failure telemetry.
• Semantic DOM Compression (50K → 800 Tokens): Developed a clustering algorithm with cross-render stable identifiers that compresses full-page contexts from ~50K tokens down to ~800 tokens, enabling sub-second, low-cost per-action model inference.
• Three-Vector Dynamic RAG Engine: Designed a self-updating RAG pipeline indexing historical scripts, extracted DOM selectors, and observed failure patterns, continually improving execution fidelity without model retraining.
• Production Full-Stack Architecture: Shipped end-to-end on AWS (Fargate, Lambda, CDK, GraphQL, Vector DB) alongside a 70K-line solo-authored frontend in TypeScript and Vue.js.