Auto Browser

> Give your AI agent a real browser, with a human in the loop.

Auto Browser is an MCP-native browser control plane for authorized workflows. It gives MCP clients, LLM agents, and operators a shared Playwright browser with human takeover, reusable auth profiles, approvals, audit trails, and local-first deployment.

Works with:

• Claude Desktop
• Cursor
• any MCP client that can talk HTTP or stdio
• direct REST callers when you want curl-first control

Why Auto Browser

• MCP-native from day one. The browser surface is already packaged as an MCP server instead of bolted on after the fact.
• Human takeover when the web gets brittle. noVNC keeps the same live session available when a person needs to step in.
• Login once, reuse later. Save named auth profiles and reopen fresh sessions that are already signed in.
• Local-first by default. Run the full stack on your own box with Docker Compose, or use Codespaces for a quick hosted demo.
• Safety rails built in. Approvals, operator identity, PII scrubbing, Witness receipts, and policy presets are all part of the product surface.
• Evidence you can hand to someone else. Witness receipt chains are Ed25519-signed, and an exported bundle verifies with `scripts/verify_witness_bundle.py` — which imports nothing from this project, so a recipient need not run or trust this controller to check it.
• We audit ourselves in public. `docs/audits/2026-08-execution-audit.md` documents an adversarial audit of this repo that found safety controls which reported success while doing nothing, with reproductions, the fixes, and the gates that close the class.
• Governed skill induction. Verified browser traces can become staged skill candidates with provenance that is signed when a mesh identity is configured — and checked on read, not just produced — plus verifier adapters and review-only graduation — agents that prove they can repeat themselves correctly, not just act once.

Release Highlights (v1.5.0)

• Read a page without paying for pixels. The new `text` observation preset returns the accessibility outline, extracted text, and interactables with no screenshot and no OCR — the cheapest way for an agent to read a page. Set `PERCEPTION_PRESET_DEFAULT=text` to make it a deployment-wide default.
• Find a string on the page in one call. `browser.find_elements` now takes a `query` (plain text or regex, case-insensitive) instead of a CSS selector and returns each match with surrounding context — no full observe needed to check one value.
• Errors agents can act on. Invalid tool arguments report field-level details, handler messages pass through instead of a generic failure, and the MCP bridge's cold-start error now says exactly how to start the controller.
• Any OpenAI-compatible model can drive the browser. A single generic adapter serves every model reachable over an OpenAI `/chat/completions` endpoint. New providers: `openrouter` (one key → ~every frontier model), `xai` (Grok), `deepseek`, `minimax`, and `openai_compatible` (custom base URL for self-hosted Ollama / vLLM / LM Studio, Azure, Together, Groq, Fireworks, …). Vision + function-calling with a content-parse fallback for endpoints that ignore `tool_choice`.
• `browser://audit/events` MCP resource. List and read recent audit events across sessions directly over MCP.
• Playwright pin parity enforced in CI. The controller (pip) and browser-node (npm) Playwright versions must match exactly — a single-side bump can no longer merge and crash-loop compose deployments.
• On PyPI. `pip install auto-browser-client` for the SDK, `pip install auto-browser-langchain` for the LangChain/LangGraph/CrewAI adapters, and `uvx auto-browser-mcp` to run the MCP stdio bridge with zero setup. Releases publish via PyPI trusted publishing (OIDC) on tag push.

Since v1.3.0

• `browser_manager.py` is now a pure facade + composition root (1,284 → 769 lines), with domain logic extracted into `app/browser/services/`.
• Fork state exports are encrypted at rest and shadow-browse state never touches disk.
• Download capture tasks can no longer be garbage-collected mid-flight, shadow-browse failures roll back cleanly, and page listeners survive object-id reuse.
• Release gates in CI enforce dependency audits, fixture evals, client tests, Python wheel builds, and the 80% controller coverage gate on Python 3.11 and 3.14.

See CHANGELOG.md for the full release history.

Good Fits

• internal dashboards and admin tools
• operator-assisted QA and browser debugging
• login-once, reuse-later account workflows
• brittle sites where a human may need to recover the flow
• MCP-powered agent workflows that need a real browser, not just HTML fetches

Not the Goal

• CAPTCHA solving
• unauthorized scraping or account automation
• deceptive identity shaping or bypass tooling

What You Get

Quickstart

git clone https://github.com/LvcidPsyche/auto-browser.git
cd auto-browser
docker compose up --build

That is enough for local development with the default settings.

Optional:

cp .env.example .env
make doctor

Run `make doctor` from a normal terminal with local Docker access and permission to open localhost sockets.

Open:

• API docs: `http://127.0.0.1:8000/docs`
• Operator dashboard: `http://127.0.0.1:8000/dashboard`
• Visual takeover: `http://127.0.0.1:6080/vnc.html?autoconnect=true&resize=scale`

All published ports bind to `127.0.0.1` by default.

Try It in Codespaces

Codespaces provisions the stack automatically. The dashboard and noVNC tabs are usually ready in about 90 seconds.

First Useful Demo

The highest-signal flow in this repo is:

1. create a session

2. log in manually if the site needs a human

3. save the session as a named auth profile

4. open a new session from that auth profile

5. continue work without reauthing

Start here:

• `examples/login-and-save-profile.md`
• `examples/README.md`

Minimal session creation:

curl -s http://127.0.0.1:8000/sessions \
  -X POST \
  -H 'content-type: application/json' \
  -d '{"name":"demo","start_url":"https://example.com"}' | jq

Minimal observation:

curl -s http://127.0.0.1:8000/sessions//observe | jq

MCP Clients

Auto Browser exposes:

• an HTTP MCP endpoint at `http://127.0.0.1:8000/mcp`
• convenience endpoints at `http://127.0.0.1:8000/mcp/tools` and `http://127.0.0.1:8000/mcp/tools/call`
• a stdio bridge: `uvx auto-browser-mcp` from PyPI, or `scripts/mcp_stdio_bridge.py` in a repo checkout

The default MCP tool profile is `curated`, which keeps the browser surface compact for better tool selection. If you want the full internal tool surface, set:

MCP_TOOL_PROFILE=full

Raw tool-call example:

curl -s http://127.0.0.1:8000/mcp/tools/call \
  -X POST \
  -H 'content-type: application/json' \
  -d '{
    "name":"browser.create_session",
    "arguments":{
      "name":"demo",
      "start_url":"https://example.com"
    }
  }' | jq

Client setup guides:

• `docs/mcp-clients.md`
• `examples/claude-desktop-setup.md`
• `examples/cursor-mcp-setup.md`
• `examples/claude_desktop_config.json`

For resource listing, resource reads, and subscription-style update examples,

see `docs/mcp-clients.md#resources-and-subscriptions`.

Convergence Harness

Auto Browser ships a Stage 0 convergence harness for Agent Skill Induction. It runs a structured task contract, records tamper-checked traces, verifies completion, and writes a staged skill candidate carrying provenance. With a mesh identity configured that provenance is signed, and the registry verifies the signature before serving a candidate — a candidate that fails the check, or that was dropped into the staging directory unsigned, is refused. Candidates induced from a mock run are marked `simulated` so they cannot pass as converged. Generated skills are staged only — promotion stays explicit and reviewed.

Read-only inspection tools (`harness.list_runs`, `harness.get_status`, `harness.get_trace`) are exposed in the default `curated` MCP tool profile so agents can introspect harness state without elevated access. Convergence runs, drift checks, candidate management, and graduation require `MCP_TOOL_PROFILE=full`, or can be invoked directly over REST.

Start with `docs/convergence-harness.md`. A deterministic local smoke is:

python -m controller.harness.run --contract evals/contracts/example_read.json --mock-final-url https://example.com --mock-final-text "Example Domain"

For MCP clients, set `MCP_TOOL_PROFILE=full` to expose the `harness.*` tools.

Security and Compliance

For a real private deployment, set at least:

APP_ENV=production
API_BIND_SCOPE=exposed
API_BEARER_TOKEN=
REQUIRE_OPERATOR_ID=true
AUTH_STATE_ENCRYPTION_KEY=
REQUIRE_AUTH_STATE_ENCRYPTION=true
REQUEST_RATE_LIMIT_ENABLED=true
METRICS_ENABLED=true
STEALTH_ENABLED=false

`COMPLIANCE_TEMPLATE` can apply a preconfigured posture at startup:

Both presets require upload approvals and enable Witness receipts. Startup writes the applied policy to `/data/compliance-manifest.json`. The legacy names (`HIPAA`, `SOC2`, `GDPR`, `PCI-DSS`) still work as deprecated aliases and emit a warning at startup.

Example:

COMPLIANCE_TEMPLATE=strict docker compose up

For deployment details, hosted Witness notes, CLI auth modes, and reverse-SSH guidance, see:

• `docs/deployment.md`
• `docs/production-hardening.md`

Architecture at a Glance

flowchart LR
    User[Human operator] -->|watch / takeover| noVNC[noVNC]
    LLM[Any model: OpenAI / Claude / Gemini / OpenRouter / Grok / DeepSeek / MiniMax / local] -->|shared tools| Controller[Controller API]
    Controller -->|Playwright protocol| Browser[Browser node]
    noVNC --> Browser
    Browser --> Artifacts[(screenshots / traces / auth state)]
    Controller --> Artifacts
    Controller --> Policy[Allowlist + approval gates]

Model providers

First-class adapters for OpenAI, Claude, and Gemini (API or CLI). Beyond those,

a single generic OpenAI-compatible adapter drives any model reachable over an OpenAI

`/chat/completions` endpoint — set an API key to enable it:

Vision (screenshots) is used for every provider except text-only ones. See `.env.example` for

the `*_API_KEY` / `*_BASE_URL` / `*_MODEL` settings.

Core components:

• `browser-node/` runs Chromium, Xvfb, x11vnc, and noVNC
• `controller/` exposes the FastAPI controller, MCP transport, policy rails, and orchestration endpoints
• `data/` holds runtime artifacts, auth state, approvals, audit logs, and optional CLI caches
• `scripts/` contains local helpers for doctor, smoke tests, bridges, and release checks

Repo Guide

Common Commands

Documentation Map

Contributing

If you want to help, start with:

• `CONTRIBUTING.md`
• `docs/good-first-issues.md`
• `CODE_OF_CONDUCT.md`

If Auto Browser is useful, a star helps other people find it. Sponsorship and tip options live in `TIPS.md`.