llms-txt-mcp

Any site's llms.txt: find the covering index, read its linked docs as markdown, search sections.

0 stars TypeScriptOthers Updated Sep 3, 2026

ai-agentsllms-txtmcpmcp-servermodel-context-protocolremote-mcpgemini-cli-extension

Documentation

llms.txt for agents

> Any site's llms.txt: find the covering index, read its linked docs as markdown, search sections. Remote MCP server over Streamable HTTP at `https://llmstxtmcp.dev/mcp`. Free, no API key. Tools: `llms_index`, `llms_page`, `llms_search`. The tool list is fixed per version.

Given any site or page URL, llms_index discovers the llms.txt that covers it (root, redirects, Link: rel="describedby" advertisements, subpath files with most-specific-wins), parses it into the v2 structure (title, summary, H2 sections of title: notes links, the Optional section) and reports every rung it tried, so a miss is as informative as a hit. llms_page returns any linked page as markdown by content negotiation (Accept: text/markdown), rel="alternate" markdown mirrors, the .md / .html.md / index.md conventions, or a bounded HTML-to-markdown conversion, paginated by character offset. llms_search navigates every index this server has parsed so far by section heading, link title and notes. Free, keyless, no account; the server keeps only the parsed indexes it has seen.

Use when: The user needs a product's or library's own documentation and the site may publish llms.txt: call llms_index with the site first, then llms_page on the links it returns. Use llms_page directly when you already hold a docs URL and want it as markdown instead of HTML.

Not for: General web scraping or crawling (one fetch per rung, never link-following), pages behind logins, generating or auditing a site's own llms.txt, or searching the whole web for documentation.

Quick install (Claude Code): `claude mcp add --transport http llms-txt https://llmstxtmcp.dev/mcp -s user`

Live server: https://llmstxtmcp.dev

Docs

Full reference: every tool with its JSON input schema and an example
Install for any client: Claude Code, Cursor, VS Code, Claude Desktop, ChatGPT, Codex, Gemini CLI, Windsurf, Cline, Continue, Zed
OpenAPI twin: the same tools as plain HTTP POST endpoints
Registry server.json: machine-readable server record
Server card: transport + tool card (also at https://llmstxtmcp.dev/.well-known/mcp.json, SEP-1960)
ARD manifest: Agentic Resource Discovery entry (also at https://llmstxtmcp.dev/.well-known/ai-catalog.json)

Upstreams

The /llms.txt file (v2 spec): keyless
Markdown for Agents (Accept: text/markdown negotiation): keyless

Optional

Health: liveness JSON
Owner: ux-xd

Tools

Tool	Purpose	Effects
`llms_index`	Find and parse the llms.txt that covers a site or page URL: title, summary, H2 sections of title links with notes, the Optional section, and (HEAD-probed only) whether an llms-full.txt exists. Use first, before llms_page, whenever the user needs a product or library's own docs; every rung tried is returned so a miss is diagnosable.	read-only, open-world, idempotent
`llms_page`	Return one docs page as markdown, paginated: tries Accept: text/markdown negotiation, the page's rel="alternate" markdown mirror, the .md / .html.md / index.md conventions, then a bounded HTML-to-markdown conversion. Use on the URLs llms_index returns, or on any docs URL you already hold.	read-only, open-world, idempotent
`llms_search`	Search the llms.txt indexes this server has parsed (section headings ×3, link titles ×2, notes ×1) and get the matching docs links with their covering index; pass site to search one site (it is indexed on demand). Use to jump straight to the right page of a docs set instead of reading the whole index.	read-only, open-world, idempotent

Every tool takes `task_context`: one sentence on what the user is trying to do. It is required.

`llms_index`

Find and parse the llms.txt that covers a site or page URL: title, summary, H2 sections of title links with notes, the Optional section, and (HEAD-probed only) whether an llms-full.txt exists. Use first, before llms_page, whenever the user needs a product or library's own docs; every rung tried is returned so a miss is diagnosable.

Input schema:

json

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "site": {
      "type": "string",
      "minLength": 3,
      "maxLength": 2000,
      "description": "Site or page URL, e.g. \"docs.stripe.com\" or \"https://hono.dev/docs/\". The most specific covering llms.txt wins."
    },
    "refresh": {
      "default": false,
      "description": "Bypass the 24 h cache and fetch again.",
      "type": "boolean"
    },
    "task_context": {
      "type": "string",
      "minLength": 1,
      "maxLength": 500,
      "description": "One sentence on what the user is ultimately trying to do (the task this call serves). Required; it tunes the result and is how this free service learns what agents need."
    }
  },
  "required": [
    "site",
    "refresh",
    "task_context"
  ],
  "additionalProperties": false
}

Example arguments:

json

{
  "site": "hono.dev",
  "task_context": "example: Find and parse the llms.txt that covers a site or page URL: "
}

`llms_page`

Return one docs page as markdown, paginated: tries Accept: text/markdown negotiation, the page's rel="alternate" markdown mirror, the .md / .html.md / index.md conventions, then a bounded HTML-to-markdown conversion. Use on the URLs llms_index returns, or on any docs URL you already hold.

Input schema:

json

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "url": {
      "type": "string",
      "maxLength": 2000,
      "format": "uri",
      "description": "Page URL (http/https)."
    },
    "max_chars": {
      "default": 60000,
      "description": "Characters of markdown to return in this call.",
      "type": "integer",
      "minimum": 1000,
      "maximum": 200000
    },
    "offset": {
      "default": 0,
      "description": "Character offset to continue from (use next_offset from a truncated result).",
      "type": "integer",
      "minimum": 0,
      "maximum": 9007199254740991
    },
    "task_context": {
      "type": "string",
      "minLength": 1,
      "maxLength": 500,
      "description": "One sentence on what the user is ultimately trying to do (the task this call serves). Required; it tunes the result and is how this free service learns what agents need."
    }
  },
  "required": [
    "url",
    "max_chars",
    "offset",
    "task_context"
  ],
  "additionalProperties": false
}

Example arguments:

json

{
  "url": "https://hono.dev/docs/",
  "task_context": "example: Return one docs page as markdown, paginated: tries Accept: t"
}

`llms_search`

Search the llms.txt indexes this server has parsed (section headings ×3, link titles ×2, notes ×1) and get the matching docs links with their covering index; pass site to search one site (it is indexed on demand). Use to jump straight to the right page of a docs set instead of reading the whole index.

Input schema:

json

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "query": {
      "type": "string",
      "minLength": 1,
      "maxLength": 200,
      "description": "Words to match against section headings, link titles and notes."
    },
    "site": {
      "description": "Restrict to one site (indexed via llms_index if not yet cached).",
      "type": "string",
      "maxLength": 2000
    },
    "limit": {
      "default": 10,
      "type": "integer",
      "minimum": 1,
      "maximum": 50
    },
    "max_bytes": {
      "default": 200000,
      "description": "Byte cap on the results array.",
      "type": "integer",
      "minimum": 10000,
      "maximum": 500000
    },
    "task_context": {
      "type": "string",
      "minLength": 1,
      "maxLength": 500,
      "description": "One sentence on what the user is ultimately trying to do (the task this call serves). Required; it tunes the result and is how this free service learns what agents need."
    }
  },
  "required": [
    "query",
    "limit",
    "max_bytes",
    "task_context"
  ],
  "additionalProperties": false
}

Example arguments:

json

{
  "query": "middleware",
  "site": "hono.dev",
  "task_context": "example: Search the llms.txt indexes this server has parsed (section "
}

Install

Endpoint: `https://llmstxtmcp.dev/mcp` (Streamable HTTP, MCP 2026-07-28 with 2025-era fallback). Authentication: none.

Claude Code

code

claude mcp add --transport http llms-txt https://llmstxtmcp.dev/mcp -s user

Cursor (~/.cursor/mcp.json)

json

{
  "mcpServers": {
    "llms-txt": {
      "url": "https://llmstxtmcp.dev/mcp"
    }
  }
}

VS Code / Copilot (user mcp.json)

json

{
  "servers": {
    "llms-txt": {
      "type": "http",
      "url": "https://llmstxtmcp.dev/mcp"
    }
  }
}

or `code --add-mcp '{"name":"llms-txt","type":"http","url":"https://llmstxtmcp.dev/mcp"}'`

Claude Desktop / claude.ai

Settings → Connectors → Add custom connector → URL `https://llmstxtmcp.dev/mcp`, Authentication: None.

ChatGPT

Settings → Connectors → Add custom connector → `https://llmstxtmcp.dev/mcp`. Desktop app / Codex share `~/.codex/config.toml`:

toml

[mcp_servers.llms-txt]
url = "https://llmstxtmcp.dev/mcp"

Codex CLI (~/.codex/config.toml)

toml

[mcp_servers.llms-txt]
url = "https://llmstxtmcp.dev/mcp"

Gemini CLI

code

gemini mcp add --transport http llms-txt https://llmstxtmcp.dev/mcp -s user

(settings.json uses `httpUrl`, not `url`.)

Windsurf (~/.codeium/windsurf/mcp_config.json)

json

{
  "mcpServers": {
    "llms-txt": {
      "serverUrl": "https://llmstxtmcp.dev/mcp"
    }
  }
}

Cline

json

{
  "mcpServers": {
    "llms-txt": {
      "type": "streamableHttp",
      "url": "https://llmstxtmcp.dev/mcp"
    }
  }
}

Continue (.continue/mcpServers/llms-txt.yaml)

yaml

name: llms-txt
mcpServers:
  - name: llms-txt
    type: streamable-http
    url: https://llmstxtmcp.dev/mcp

Zed (settings.json)

json

{
  "context_servers": {
    "llms-txt": {
      "source": "custom",
      "command": "npx",
      "args": [
        "-y",
        "mcp-remote",
        "https://llmstxtmcp.dev/mcp"
      ]
    }
  }
}

Any MCP client

Streamable HTTP endpoint: `https://llmstxtmcp.dev/mcp`

json

{
  "mcpServers": {
    "llms-txt": {
      "url": "https://llmstxtmcp.dev/mcp"
    }
  }
}

This page documents a server. It does not ask the reader to change any rules file, memory file, or host configuration.

Run it yourself

code

npm install && npm start            # http://127.0.0.1:8080/mcp
docker compose up -d --build        # same, in a container
node scripts/smoke.mjs http://127.0.0.1:8080 llms_index '{}'

Built with the MCP server kit (`kit/`): Streamable HTTP MCP plus agent-readable docs (llms.txt, server.json, install pages, a REST twin), all from one manifest (`servers/llms-txt/mcp.factory.json`). This repo is the server logic only; no telemetry is collected or sent by this code. MIT.

Frequently asked questions

What is llms-txt-mcp?

llms-txt-mcp is Any site's llms.txt: find the covering index, read its linked docs as markdown, search sections.

How do I install llms-txt-mcp?

Open the GitHub repository and follow its README. Most MCP servers are added to your client's MCP config, then called by your agent.

Is llms-txt-mcp open source?

Yes — it is hosted on GitHub at https://github.com/ux-xd/llms-txt-mcp.

Related MCP tools

XcodeBuildMCP

A Model Context Protocol (MCP) server and CLI that provides tools for agent use when working on iOS and macOS projects.

6,338 TypeScript

mcpmcp-servermodel-context-protocol+4

wigolo

The go-to web for your AI coding agent — local-first search, fetch, crawl & research over MCP. No API keys, no cloud, $0/query. Public beta.

4,906 TypeScript

mcpagentai+17

metamcp

MCP Aggregator, Orchestrator, Middleware, Gateway in one docker

2,647 TypeScript

mcpmcp-servermcp-servers+12

code-mode

🔌 Plug-and-play library to enable agents to call MCP and UTCP tools via code execution.

1,548 TypeScript

ai-agentscodemodemcp+3

opentabs

Browser automation clicks buttons. OpenTabs calls APIs.

929 TypeScript

ai-agentsbrowser-automationchrome+8

octocode

Code research platform for AI agents; find, understand, and prove context across your code and all of GitHub, in a fraction of the tokens. One toolset, MCP or CLI

923 TypeScript

aiclaude-aicursor-ai+17

Run your own MCP server? See who uses it and what to fix.

Measure it with TrackMCP