mcpscope

**A local-first workbench for developing, inspecting, and benchmarking MCP servers against local

(LM Studio, Ollama) or remote (OpenRouter) models — one shared workspace with a Web UI for you and

a CLI + MCP interface for your coding agent.**

**Its runtime ships separately as `mcpscope-engine` — an embeddable,

zero-dependency TypeScript chat/agent engine (MCP tool calling, local-first LLM providers, durable

`node:sqlite` sessions, streaming transparency events) for building your own MCP-powered app. To

start from a working app instead of a blank file, fork

mcpscope-chat-template — an MCP server, the

engine, and an embeddable chat UI in one small Node process.**

Building a good MCP server is empirical work, and you rarely do it alone anymore: a coding agent

can run every benchmark and read every trace, but the numbers only get *good* when the developer

stays in the loop — looking at the one run that failed, tightening a tool description or a rubric,

deciding what to try next, and re-running. mcpscope is built for that partnership. It is not a

benchmark harness an agent runs unattended, and not a GUI a developer drives by hand: every

capability is exposed through a web UI shaped for a person and through a CLI and an MCP interface

shaped for an agent, all over one shared data model. The human can see exactly what the agent did,

the agent can inspect exactly what the human did, and a type-tagged ID system lets both point at

the same session, turn, or tool call.

Why that loop matters: an LLM is only as good as the tools it is given — a lesson I learned

building my own MCP servers for data analytics and statistics. My first versions did the obvious

thing: wrap the existing API and hand the raw data back to the model. That turned out to be the

worst possible design — token-hungry, imprecise, and dependent on a large model with a large

context window just to produce something useful. Results only became good when the server did the

work itself — the calculations, aggregation, and filtering — and returned answers rather than

data. That experience is the premise mcpscope is built on: a good MCP server is not an API

wrapper. It is a user interface built for an LLM, designed to solve a specific job efficiently

even on a small local model. And it is an application like any other, so it starts from clear use

cases and quality criteria, and most of ordinary software engineering applies when you design and

build it.

The part that does not carry over is testing. You cannot pin end-to-end behavior down with

deterministic assertions, because the model is non-deterministic and both its inputs and outputs

are mostly natural language. So, like the wider evaluation ecosystem, mcpscope measures quality

statistically — run a set of prompts many times against a chosen model and MCP server, then read

back per-tool reliability and token cost, with optional scoring of answer quality by a separate

judge model. What mcpscope adds is keeping that measurement *inside* the iteration loop rather

than at the end of it: every run stays inspectable down to each reasoning step, tool call, and

token of context, so "the score dropped" immediately becomes "look at what the model did here."

You run a prompt, or a repeatable benchmark, against a local (LM Studio, Ollama) or remote

(OpenRouter) model, watch every reasoning step, tool call, and token of context, then change one

thing (a tool description, a parameter, an output payload) and run it again. Everything stays on

your machine.

Get started

Prerequisite (2 minutes): a running LLM backend — local

(LM Studio, Ollama) or remote

(OpenRouter). For LM Studio: open the Developer tab → **Start

server**, load a model, and note its model id — the server URL is `http://localhost:1234/v1`.

Desktop app — the easiest way to try mcpscope

Download the installer for your OS from the

**Releases page** (macOS `.dmg` — Apple

Silicon only, Windows `.exe`, Linux `AppImage`/`.deb`/`.rpm`). Everything is bundled — launch it and the workbench

opens; data lives in `~/.mcpscope`. While it runs it serves the same backend as `mcpscope serve`

at `http://localhost:3066` (MCP interface at `/mcp`), so coding agents and other MCP clients

can connect to it directly — set the `BACKEND_HOST` / `BACKEND_PORT` environment variables before

launching to change the address (shown in the app under Configuration → Server). The builds

are unsigned for now, so macOS Gatekeeper / Windows SmartScreen warn on first run.

npm — for developing an MCP server (adds the CLI)

If you're building an MCP server you'll want the CLI (and the MCP interface for your coding

agent). Requires Node.js 24+:

npm install -g mcpscope
mcpscope serve

Or run it without installing: `npx mcpscope serve`.

`mcpscope serve` starts mcpscope at `http://localhost:3066` and opens it in your browser. Data

is stored in `~/.mcpscope`; stop with `Ctrl-C`. Flags: `--port `, `--host `,

`--data-dir `, `--no-open`.

First steps (either install)

1. In the Web UI, open Configuration and add an LM connection (the base URL above) and a model config (the model id you loaded), then set it as the default model. No MCP profile is needed to start — mcpscope ships zero-setup companion MCP servers. Prefer editing a file? The same setup as JSON: CONFIG.md.

2. Create a session, select a companion server (e.g. Open-Meteo Weather), send a prompt, and inspect the full trace: setup, tool definitions, reasoning, tool calls and results, and a color-coded context breakdown per turn.

3. Add an MCP server profile when you are ready to point mcpscope at your own server.

4. Define a benchmark and run it to test an MCP server repeatably across models — then hand the same loop to your coding agent.

The full walkthrough is in TUTORIAL.md; the worked example with real numbers is

EXAMPLE.md.

For coding agents

mcpscope speaks MCP itself — connect your agent and it can drive the whole loop on the same

sessions you see in the UI:

claude mcp add --transport http mcpscope http://localhost:3066/mcp

{ "mcpServers": { "mcpscope": { "type": "http", "url": "http://localhost:3066/mcp" } } }

Then a prompt like this is enough to put the agent to work:

> mcpscope is running at localhost:3066 (MCP at /mcp; the `mcpscope` CLI is the identical

> surface). Use its `mcpscope_*` tools to benchmark my MCP server: start with

> `mcpscope_list_mcp_profiles` and `mcpscope_list_model_configs`, create sessions and send

> prompts with `wait: true` so you never poll, and follow the loop in EXAMPLE.md.

Every operation is both a CLI command and an MCP tool (test-enforced parity), results are

snake_case JSON, and `create`/`send` take `wait` so agents get terminal results in one call.

Details: MCP.md.

Other ways to run

• Docker: a released image is published to GHCR. See TUTORIAL.md for the step-by-step path and RELEASING.md for image tags.
• From source: for working on mcpscope itself, see DEVELOPMENT.md.

What you can do

• Inspect sessions: watch how a model reads tool definitions, reasons, calls tools, and consumes the context window, with auditable token attribution per part.
• Benchmark MCP servers: a reusable suite of prompts run N× against a chosen model and MCP server, producing a per-tool error/usage scorecard and per-case reliability (pass@k / pass^k).
• LLM-evaluate answer quality: a separate judge model scores each run against a per-case rubric (see BENCHMARK.md).
• Drive it from the shell or as MCP tools: every operation is both a `mcpscope ` CLI command and a `mcpscope_` MCP tool, so a coding agent can run the whole loop — on the same sessions you see in the UI.

How it compares

Plenty of good tools touch parts of this space; the difference is which part of the loop they

serve, and who they keep in it.

• MCP Inspector pokes an MCP server at the protocol level — list tools, call them by hand. There is no model in the loop; mcpscope tests what a model actually *does* with your server.
• mcpsnoop captures real client↔server MCP traffic on the wire. mcpscope runs the model itself and attributes every token of context.
• MCPJam is the closest neighbor: an LLM playground with evals and traces. mcpscope differs on per-part token attribution, pass@k / pass^k reliability statistics, and the human+agent parity over one shared store.
• promptfoo, mcp-eval, and DeepEval are config- or code-driven eval harnesses, at their best as unattended regression gates in CI. mcpscope is the interactive workbench for the iteration loop *before* that — and exposes the same operations over CLI/MCP so an agent can drive it too.
• MCPBench, MCP-Bench, and MCPMark benchmark *models* against fixed sets of servers. mcpscope benchmarks *your server* against the models you choose.
• LM Studio and Open WebUI are chat interfaces with tool support — for the human only, with aggregate token counts at best. LM Studio is one of mcpscope's supported backends, not a competitor.

If one of these fits your workflow better, use it — several are excellent. mcpscope is for the

loop where you and your coding agent iterate on your own MCP server together.

Documentation

Getting started

• TUTORIAL.md - install, configure, run a session, and benchmark an MCP server
• EXAMPLE.md - the worked example: inspect → benchmark → change one thing → watch the metric move
• CONFIG.md - the `mcpscope.config.json` reference, including fully headless setup
• COMPANIONS.md - bundled zero-config companion MCP servers you can select without any setup
• BENCHMARK.md - benchmark suite/case/run model, deterministic metrics, and LLM rubric evaluation

Interfaces

• MCP.md - MCP interface: transport, tool surface, and structured results
• CLI.md - CLI commands, flags, output format, and exit codes
• EMBEDDING.md - embed the `mcpscope-engine` package in your own Node.js/TypeScript app: `createEngine()`, config, sessions, events, and a full Express integration
• mcpscope-chat-template - a forkable starter app built on the engine: MCP server + agent loop + embeddable chat UI in one Node process, with docs on MCP tool design

Internals & contributing

• DEVELOPMENT.md - run from source, build, dev helpers, and repository notes
• AGENTS.md - guide for AI coding agents working on mcpscope itself: project shape, parity principle, working style, validation
• ARCHITECTURE.md - system design, persistence, streaming, replay, and API surface
• PROVIDERS.md - provider internals (LM Studio, Ollama, OpenRouter): reasoning tokens, token counting, context windows, model loading/unloading (incl. auto-swap)
• DATA-MODEL.md - canonical runtime tree, part taxonomy, and IDs
• DATABASE-SCHEMA.md - SQLite tables, foreign keys, and ER diagram
• DESIGN-SYSTEM.md - frontend design system: the tokens, primitives, and patterns that keep the GUI consistent
• design-assets/ - master logo SVGs (logo, mark, wordmark, favicon); see its README
• TESTING.md - test strategy, replay, and how to add regressions
• RELEASING.md - tag-driven release workflow: npm publish, GHCR image, and desktop installers

Internal/contributor docs live under `docs/`; user-facing guides stay at the repository root.