🌊🔍 Lenses MCP Server for Apache Kafka 🔎🌊

This is the Lenses MCP (Model Context Protocol) server for Apache Kafka. Lenses offers a developer experience solution for engineers building real-time applications connected to Kafka. It's built for the enterprise and backed by a powerful IAM and governance model.

With Lenses, you can find, explore, transform, integrate and replicate data across a multi-Kafka and vendor estate. Now, all this power is accessible through your AI tools and AI Agents via MCP, bringing real-time context into your agentic engineering workflows.

The quickest way to try the MCP server is with the free Lenses Community Edition, which runs Lenses MCP Server as a remote MCP server and comes with a pre-configured single broker Kafka cluster with demo data, ideal for local development or evaluation (steps here).

Table of Contents

• 1. Install uv and Python
• 2. Configure Environment Variables
• 3. OAuth 2.1 Authentication (Recommended)
• 4. Lenses API Key (Fallback)
• 5. Running the Server Locally
• 6. Running with Docker
• 7. OpenTelemetry Tracing
• 8. Optional Context7 MCP Server
• Appendix: OAuth Flow Details

1. Install uv and Python

We use `uv` for dependency management and project setup. If you don't have `uv` installed, follow the official installation guide.

This project requires *Python 3.13* (any 3.13.x release). To verify your Python version, run:

uv run python --version

2. Configure Environment Variables

Copy the example environment file and configure it based on your authentication method:

cp .env.example .env

Required variables depend on your authentication choice:

• For OAuth (recommended): `LENSES_URL` and `MCP_ADVERTISED_URL`
• For API Key (fallback): `LENSES_URL` and `LENSES_API_KEY`

3. OAuth 2.1 Authentication (Recommended)

OAuth 2.1 is the recommended authentication method for all Lenses MCP deployments. It provides secure, scope-based authorization without sharing static API keys.

How it works

OAuth 2.1 uses bearer tokens that are validated via RFC 7662 Token Introspection. The flow involves three participants:

1. MCP Client — Your AI tool (Claude, Cursor, etc.)

2. Authorization Server — Lenses HQ at `LENSES_ADVERTISED_URL`

3. MCP Server — This server (the resource server)

When you connect, the client automatically:

1. Discovers OAuth metadata from this server (`/.well-known/oauth-protected-resource/mcp`)

2. Registers itself with the authorization server

3. Initiates OAuth authorization (with PKCE) and gets an access token

4. Uses the token to authenticate requests to this MCP server

This server then validates the token with Lenses HQ before allowing access to Kafka resources.

Simple setup

To use OAuth, you should set `OAUTH_ENABLED` to `true`, then you only need to set two environment variables:

OAUTH_ENABLED=true
LENSES_URL=https://lenses.example.com
MCP_ADVERTISED_URL=http://localhost:8000

• `LENSES_URL` — Your Lenses instance (used internally and as the OAuth authorization server)
• `MCP_ADVERTISED_URL` — The public URL where this MCP server is reachable by clients

`TRANSPORT` automatically defaults to `http` when `MCP_ADVERTISED_URL` is set.

Advanced: split-plane deployments

If the MCP server reaches Lenses on an internal address but clients reach it on a public URL:

OAUTH_ENABLED=true
LENSES_URL=http://lenses-hq.internal:9991
LENSES_ADVERTISED_URL=https://lenses.example.com
MCP_ADVERTISED_URL=https://mcp.example.com

Authorization scopes

The server advertises three scopes:

When you authenticate, you'll be prompted to grant these scopes. Your token will only grant the scopes you select.

Lenses HQ configuration

Lenses HQ must support OAuth 2.0 and token introspection. Ensure your Lenses HQ config includes:

oauth2:
  authorizationServer:
    unauthenticatedIntrospection: true

This allows the MCP server to validate tokens without client credentials.

4. Lenses API Key (Fallback)

For backward compatibility and testing, you can use a static API key instead of OAuth. This is not recommended for production but may be useful for local development or legacy systems.

Create a Lenses API key by provisioning an IAM Service Account in Lenses. Add the API key to `.env`:

LENSES_URL=https://lenses.example.com
LENSES_API_KEY=

When using API key authentication, `TRANSPORT` defaults to `stdio` (local only) unless you explicitly set `MCP_ADVERTISED_URL`.

5. Running the Server Locally

First, install dependencies:

uv sync

With OAuth (Recommended)

Run with stdio transport (for local AI tools):

OAUTH_ENABLED=true \
LENSES_URL=https://lenses.example.com \
MCP_ADVERTISED_URL=http://localhost:8000 \
uv run src/lenses_mcp/server.py

Or run with HTTP transport (for remote clients):

OAUTH_ENABLED=true \
LENSES_URL=https://lenses.example.com \
MCP_ADVERTISED_URL=http://localhost:8000 \
uv run fastmcp run src/lenses_mcp/server.py --transport=http --port=8000

To configure in Claude Desktop, Cursor, or similar tools:

{
  "mcpServers": {
    "Lenses": {
      "command": "uv",
      "args": [
        "run",
        "--project", "",
        "--with", "fastmcp",
        "fastmcp",
        "run",
        "/src/lenses_mcp/server.py"
      ],
      "env": {
        "OAUTH_ENABLED": "true",
        "LENSES_URL": "https://lenses.example.com",
        "MCP_ADVERTISED_URL": "http://localhost:8000"
      },
      "transport": "stdio"
    }
  }
}

With API Key (Legacy)

Using a static API key:

LENSES_URL=https://lenses.example.com \
LENSES_API_KEY= \
uv run src/lenses_mcp/server.py

Or with HTTP transport:

LENSES_URL=https://lenses.example.com \
LENSES_API_KEY= \
uv run fastmcp run src/lenses_mcp/server.py --transport=http --port=8000

To configure in Claude Desktop, Cursor, or similar tools:

{
  "mcpServers": {
    "Lenses.io": {
      "command": "uv",
      "args": [
        "run",
        "--project", "",
        "--with", "fastmcp",
        "fastmcp",
        "run",
        "/src/lenses_mcp/server.py"
      ],
      "env": {
        "LENSES_URL": "https://lenses.example.com",
        "LENSES_API_KEY": ""
      },
      "transport": "stdio"
    }
  }
}

Note: Some clients may require the absolute path to `uv` in the command.

6. Running with Docker

The Lenses MCP server is available as a Docker image at `lensesio/mcp`. You can run it with OAuth (recommended) or API key authentication.

With OAuth (Recommended)

Stdio transport (for local AI tools):

docker run --rm -it \
  -e OAUTH_ENABLED=true \
  -e LENSES_URL=https://lenses.example.com \
  -e MCP_ADVERTISED_URL=http://localhost:8000 \
  lensesio/mcp

HTTP transport (for remote clients, listens on `http://0.0.0.0:8000/mcp`):

docker run --rm -it -p 8000:8000 \
  -e OAUTH_ENABLED=true \
  -e LENSES_URL=https://lenses.example.com \
  -e MCP_ADVERTISED_URL=http://localhost:8000 \
  -e TRANSPORT=http \
  lensesio/mcp

For split-plane deployments where the MCP server reaches Lenses internally but clients use a public URL:

docker run --rm -it -p 8000:8000 \
  -e OAUTH_ENABLED=true \
  -e LENSES_URL=http://lenses-hq.internal:9991 \
  -e LENSES_ADVERTISED_URL=https://lenses.example.com \
  -e MCP_ADVERTISED_URL=https://mcp.example.com \
  -e TRANSPORT=http \
  lensesio/mcp

With API Key (Legacy)

Stdio transport (for local AI tools):

docker run --rm -it \
  -e LENSES_API_KEY= \
  -e LENSES_URL=https://lenses.example.com \
  lensesio/mcp

HTTP transport (for remote clients, listens on `http://0.0.0.0:8000/mcp`):

docker run --rm -it -p 8000:8000 \
  -e LENSES_API_KEY= \
  -e LENSES_URL=https://lenses.example.com \
  -e TRANSPORT=http \
  lensesio/mcp

Environment Variables Reference

Legacy environment variables (for backward compatibility):

• `LENSES_API_HTTP_URL`, `LENSES_API_HTTP_PORT`
• `LENSES_API_WEBSOCKET_URL`, `LENSES_API_WEBSOCKET_PORT`

These are automatically derived from `LENSES_URL` but can be explicitly set to override.

Transport Endpoints

• stdio: Standard input/output (no network endpoint)
• http: HTTP endpoint at `/mcp`
• sse: Server-Sent Events endpoint at `/sse`

Building the Docker Image Locally

To build the Docker image locally:

docker build -t lensesio/mcp .

7. OpenTelemetry Tracing

The server is built on FastMCP 4, which emits an OpenTelemetry span for every

MCP request — `tools/call `, `tools/list`, `prompts/get `

and so on. Spans carry the tool or prompt name, the MCP session id, the

negotiated protocol version, and error status when a call fails.

Tracing is off by default and costs nothing until you switch it on.

Enabling

OTEL_ENABLED=true \
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 \
uv run python src/lenses_mcp/server.py

With Docker:

docker run -p 8000:8000 \
   -e LENSES_URL= \
   -e LENSES_API_KEY= \
   -e TRANSPORT=http \
   -e OTEL_ENABLED=true \
   -e OTEL_EXPORTER_OTLP_ENDPOINT=http://collector:4318 \
   lensesio/mcp

`OTEL_EXPORTER_OTLP_ENDPOINT` is a base URL — the SDK appends `/v1/traces`

for HTTP. Point it at the collector root, not at the traces path. Use

`OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` if you need to specify the full URL.

Exporters

The OTLP HTTP exporter (port 4318) is included by default. The gRPC

exporter (port 4317) pulls in `grpcio` and adds roughly 27MB to the image, so

it is an optional extra:

uv sync --extra otlp-grpc
# then
OTEL_EXPORTER_OTLP_PROTOCOL=grpc OTEL_EXPORTER_OTLP_ENDPOINT=http://collector:4317 ...

Asking for `grpc` without the extra installed logs a warning and leaves tracing

off; it never prevents the server from starting. The same is true of any other

telemetry misconfiguration — an unreachable collector or a bad exporter name is

logged and the server carries on serving requests.

Verifying locally

The quickest check needs no collector — print spans to stderr:

OTEL_ENABLED=true OTEL_EXPORTER=console uv run python src/lenses_mcp/server.py

The console exporter writes to stderr, not stdout: with the default `stdio`

transport, stdout carries the JSON-RPC wire and anything else printed there

would corrupt it. Redirect stderr to a file if the spans crowd your terminal:

OTEL_ENABLED=true OTEL_EXPORTER=console \
uv run python src/lenses_mcp/server.py 2>spans.log

For a trace UI, Jaeger accepts OTLP/HTTP on 4318 and serves its UI on 16686:

docker run -p 16686:16686 -p 4318:4318 \
  -e COLLECTOR_OTLP_ENABLED=true jaegertracing/all-in-one:1.62.0

Then browse and select the `lenses-mcp` service.

Trace context propagation

Spans are emitted per MCP request; there is no session-level parent span. A

nested trace only forms when the client propagates trace context

(`traceparent` in the request `_meta`), which FastMCP extracts and parents

from. Clients that are not OpenTelemetry-instrumented produce one root trace

per call, which is expected. To get grouped traces, instrument the agent that

calls this server.

8. Optional Context7 MCP Server

Lenses documentation is available on Context7. It is optional but highly recommended to use the Context7 MCP Server and adjust your prompts with `use context7` to ensure the documentation available to the LLM is up to date.

Appendix: OAuth Flow Details

Token validation sequence

The MCP server validates bearer tokens using the following sequence:

1. Protected Resource Metadata (RFC 9728) — `RemoteAuthProvider` serves `/.well-known/oauth-protected-resource/mcp` so clients can discover which authorization server to use and what scopes are available.

2. Auto-Discovery — On the first incoming request, the `DiscoveryTokenVerifier` lazily fetches `{LENSES_ADVERTISED_URL}/.well-known/oauth-authorization-server` to discover the `introspection_endpoint`. The endpoint URL can also be set explicitly via `INTROSPECTION_URL`.

3. Token Introspection (RFC 7662) — For each incoming bearer token, the verifier POSTs to the introspection endpoint (`/oauth2/introspect`) without client authentication. The authorization server responds with:

Inactive or expired tokens are rejected before reaching the Lenses API.

4. Token Forwarding — Valid tokens are forwarded to the Lenses API via `Authorization: Bearer ` so Lenses can perform its own authorization checks.

Authorization scopes

The server advertises three scopes in its protected-resource metadata:

Scopes are not enforced globally at the introspection level — a token with any subset of these scopes is accepted. Per-tool scope enforcement can be added using FastMCP's `require_scopes` decorator.

Configuration and Requirements

In a simple deployment, only two environment variables are required:

LENSES_URL=https://lenses.example.com
MCP_ADVERTISED_URL=http://localhost:8000

For split-plane deployments where the MCP server reaches Lenses on an internal address but clients use a public URL, set:

LENSES_URL=http://lenses-hq.internal:9991
LENSES_ADVERTISED_URL=https://lenses.example.com
MCP_ADVERTISED_URL=https://mcp.example.com

Lenses HQ must support:

• OAuth 2.0 Authorization Server Metadata (RFC 8414) at `/.well-known/oauth-authorization-server`
• Token Introspection (RFC 7662) at the `introspection_endpoint`, with client authentication disabled
• PKCE with S256 (RFC 7636) for client authorization flows

The MCP server does not send client credentials when introspecting. Lenses HQ must be configured with:

oauth2:
  authorizationServer:
    unauthenticatedIntrospection: true

Without this setting, every bearer token will be rejected as invalid.