GovData MCP

A Model Context Protocol (MCP) server that lets AI agents search, explore, and fetch open datasets from official government data portals. One unified tool interface across the United States, United Kingdom, Canada, and Australia — with per-portal quirks (API migrations, auth walls, bilingual metadata, stale DataStore flags) absorbed behind a common adapter layer.

Built for the typical agent workflow: *"Is there a government dataset for X?"* → find candidates → inspect the files → preview rows or grab a download link and move on.

Key Features

• Four national portals, one interface. US (data.gov v4 / DCAT), UK (data.gov.uk / CKAN), Canada (open.canada.ca / CKAN), Australia (data.gov.au / CKAN + DataStore).
• Token-frugal by design. Descriptions are HTML-stripped and truncated, search results are compact summaries, and large files are never inlined by accident.
• Row-level preview. Uses CKAN DataStore where available (Australia), and transparently falls back to reading the head of CSV/TSV files everywhere else.
• Agent-friendly failure modes. Errors are returned as actionable text ("use `get_download_link` instead", "retry with `dataset_id`", "available portals: …") — never raw tracebacks.
• Extensible registry. Adding another CKAN-based portal is a single config entry; non-CKAN portals plug in via a small adapter class.
• Dual transport. `stdio` for local clients (Claude Desktop etc.), stateless `streamable-http` for serverless deployment (Google Cloud Run).

Supported Portals

Requirements

• Python 3.11+
• An api.data.gov API key for the US portal (free; other portals need no credentials)

Installation

git clone https://github.com/YOUR_ORG/Gov-Stat-MCP-Server.git
cd Gov-Stat-MCP-Server

# pip
pip install -e ".[dev]"

# or conda
conda env create -f environment.yml
conda activate opendata-mcp

Copy `.env.example` to `.env` and set your key:

DATAGOV_API_KEY=your-api-data-gov-key

Getting Started

Standard config for most MCP clients (stdio transport):

{
  "mcpServers": {
    "govdata": {
      "command": "govdata-mcp",
      "env": {
        "DATAGOV_API_KEY": "your-api-data-gov-key"
      }
    }
  }
}

Claude Code

claude mcp add govdata -e DATAGOV_API_KEY=your-key -- govdata-mcp

Claude Desktop

Follow the MCP install guide and use the standard config above. Note: `govdata-mcp` must be on the PATH Claude Desktop uses — provide an absolute path to the executable inside your virtualenv/conda env if needed, e.g. `/path/to/envs/mcp-env/bin/govdata-mcp`.

Cursor / Windsurf / other JSON-config clients

Use the standard config above in the client's MCP settings file.

HTTP transport (remote / long-running server)

Run the server in HTTP mode:

MCP_TRANSPORT=http govdata-mcp

Then point your client at the endpoint:

{
  "mcpServers": {
    "govdata": {
      "url": "http://localhost:8080/mcp"
    }
  }
}

Tools

The intended agent flow: `list_portals` → `search_datasets` → `get_dataset` → `preview_resource` / `fetch_resource` / `get_download_link`.

Discovery

• list_portals
  • Description: List available government open-data portals and their capabilities.
  • Parameters: None
  • Read-only: true

• search_datasets
  • Description: Full-text search for datasets on a portal. Returns compact summaries with dataset ids.
  • Parameters:
    • `portal` (string): Portal code — `us`, `uk`, `ca`, `au`
    • `query` (string): Search terms, e.g. `"air quality monitoring"`
    • `limit` (number, optional): Max results (default 10, max 50)
    • `org` (string, optional): Publisher/organization slug filter (CKAN portals)
    • `format` (string, optional): Resource format filter, e.g. `"CSV"` (CKAN portals)
  • Read-only: true

Exploration

• get_dataset
  • Description: Full dataset metadata — organization, license, tags, description, and the list of downloadable resources with their ids, formats, and sizes.
  • Parameters:
    • `portal` (string): Portal code
    • `dataset_id` (string): Dataset id/slug from `search_datasets`
  • Read-only: true

• preview_resource
  • Description: Preview rows from a tabular resource. Tries the portal DataStore first (row counts + text filtering); falls back to reading the head of CSV/TSV files. Non-tabular formats can't be previewed.
  • Parameters:
    • `portal` (string): Portal code
    • `resource_id` (string): Resource id from `get_dataset`
    • `rows` (number, optional): Rows to return (default 20, max 100)
    • `query` (string, optional): Full-text row filter (DataStore-backed resources only)
    • `dataset_id` (string, optional): Parent dataset id — recommended; required on `uk`
  • Read-only: true

Retrieval

• fetch_resource
  • Description: Download a small text resource (CSV/JSON/XML/…) and return its content inline, truncated at a size cap on a clean line boundary. Binary content is refused with a pointer to the download URL.
  • Parameters:
    • `portal` (string): Portal code
    • `resource_id` (string): Resource id from `get_dataset`
    • `max_kb` (number, optional): Max kilobytes to inline (default 256, cap 512)
    • `dataset_id` (string, optional): Parent dataset id — recommended; required on `uk`
  • Read-only: true

• get_download_link
  • Description: Return the direct download URL, format, and size for a resource so the agent can proceed on its own. Works for any format, including large/binary files.
  • Parameters:
    • `portal` (string): Portal code
    • `resource_id` (string): Resource id from `get_dataset`
    • `dataset_id` (string, optional): Parent dataset id — recommended; required on `uk`
  • Read-only: true

Configuration

All settings are environment variables (or a local `.env` file):

Docker

make docker-build          # build image
make docker-run            # run on :8080, exactly as Cloud Run would

The image runs the HTTP transport in stateless mode and is ready for serverless platforms (Google Cloud Run deployment script included under `deploy/`).

Development

make run          # stdio mode
make run-http     # HTTP mode on :8080
make inspect      # MCP Inspector interactive UI (requires Node.js)
make test         # offline unit tests
make test-live    # live end-to-end tests against all four real portals
make lint         # ruff check + format check

The live suite (`pytest -m live`) verifies the full search → explore → preview → fetch chain per country plus portal-specific behavior (Canada's bilingual normalization, UK's dataset-scoped resource lookup, Australia's DataStore paths). Run it before deploying — government portals change without notice.

Architecture

src/govdata_mcp/
├── server.py            # FastMCP instance + tool registration
├── config.py            # env-driven settings
├── ckan/                # generic async CKAN client, models, typed errors
├── portals/
│   ├── registry.py      # portal registry — single source of truth
│   ├── base.py          # Portal config + default (vanilla CKAN) adapter
│   ├── us.py            # data.gov v4 DCAT adapter (auth, cursor paging)
│   ├── uk.py            # dataset-scoped resource lookup
│   └── ca.py            # bilingual metadata normalization
├── tools/               # MCP tools: search, explore, fetch
└── utils/               # formatting (token economy), CSV-head preview

Adding a portal: for a standard CKAN portal, add one `Portal(...)` entry to `portals/registry.py` — done. For portals with quirks, subclass `PortalAdapter` and override the narrow hooks (`normalize_dataset`, `build_search_params`) or, for non-CKAN APIs, the operation methods themselves (see `us.py` for a full custom adapter).

TODO / Roadmap

• [ ] 🇨🇳 China portal support — no unified national open-data API exists; planned as a custom adapter targeting National Bureau of Statistics data (via an existing stats package) with graceful degradation.
• [ ] 🇯🇵 Japan portal support — e-Gov Data Portal (data.e-gov.go.jp) is CKAN-compatible and should slot into the existing registry; the richer e-Stat statistics API (app-ID auth, Japanese-language metadata) planned as a dedicated adapter.
• [ ] Offline unit test suite with recorded portal fixtures (respx) for CI
• [ ] Google Cloud Run deployment walkthrough (service is container-ready; script in `deploy/`)
• [ ] US v4 publisher/format search filters (pending API documentation)
• [ ] More CKAN portals (New Zealand data.govt.nz is a near-free addition)
• [ ] Content-type sniffing to rescue mislabeled resource formats

Security Notes

• This server is read-only toward the portals — no tool can create, modify, or delete anything.
• `fetch_resource` and `preview_resource` download from URLs contained in portal metadata; content is size-capped and binary-checked before being returned to the model, but treat fetched content as untrusted input.
• The server itself has no authentication. For remote deployment, put it behind an authenticating proxy or platform-level auth (e.g. Cloud Run with `--no-allow-unauthenticated`).
• Keep `DATAGOV_API_KEY` in `.env` / secret manager — never commit it.

License

MIT