doc-lib-mcp MCP server

A Model Context Protocol (MCP) server for document ingestion, chunking, semantic search, and note management.

Components

Resources

• Implements a simple note storage system with:
  • Custom `note://` URI scheme for accessing individual notes
  • Each note resource has a name, description, and `text/plain` mimetype

Prompts

• Provides a prompt:
  • summarize-notes: Creates summaries of all stored notes
    • Optional "style" argument to control detail level (brief/detailed)
    • Generates prompt combining all current notes with style preference

Tools

The server implements a wide range of tools:

• add-note: Add a new note to the in-memory note store
  • Arguments: `name` (string), `content` (string)
• ingest-string: Ingest and chunk a markdown or plain text string provided via message
  • Arguments: `content` (string, required), `source` (string, optional), `tags` (list of strings, optional)
• ingest-markdown: Ingest and chunk a markdown (.md) file
  • Arguments: `path` (string)
• ingest-python: Ingest and chunk a Python (.py) file
  • Arguments: `path` (string)
• ingest-openapi: Ingest and chunk an OpenAPI JSON file
  • Arguments: `path` (string)
• ingest-html: Ingest and chunk an HTML file
  • Arguments: `path` (string)
• ingest-html-url: Ingest and chunk HTML content from a URL (optionally using Playwright for dynamic content)
  • Arguments: `url` (string), `dynamic` (boolean, optional)
• smart_ingestion: Extracts all technically relevant content from a file using Gemini, then chunks it using robust markdown logic.
  • Arguments:
    • `path` (string, required): File path to ingest.
    • `prompt` (string, optional): Custom prompt to use for Gemini.
    • `tags` (list of strings, optional): Optional list of tags for classification.
  • Uses Gemini 2.0 Flash 001 to extract only code, configuration, markdown structure, and technical definitions (no summaries or commentary).
  • Passes the extracted content to a mistune 3.x-based chunker that preserves both code blocks and markdown/narrative content as separate chunks.
  • Each chunk is embedded and stored for semantic search and retrieval.
• search-chunks: Semantic search over ingested content
  • Arguments:
    • `query` (string): The semantic search query.
    • `top_k` (integer, optional, default 3): Number of top results to return.
    • `type` (string, optional): Filter results by chunk type (e.g., `code`, `html`, `markdown`).
    • `tag` (string, optional): Filter results by tag in chunk metadata.
  • Returns the most relevant chunks for a given query, optionally filtered by type and/or tag.
• delete-source: Delete all chunks from a given source
  • Arguments: `source` (string)
• delete-chunk-by-id: Delete one or more chunks by id
  • Arguments: `id` (integer, optional), `ids` (list of integers, optional)
  • You can delete a single chunk by specifying `id`, or delete multiple chunks at once by specifying `ids`.
• update-chunk-type: Update the type attribute for a chunk by id
  • Arguments: `id` (integer, required), `type` (string, required)
• ingest-batch: Ingest and chunk multiple documentation files (markdown, OpenAPI JSON, Python) in batch
  • Arguments: `paths` (list of strings)
• list-sources: List all unique sources (file paths) that have been ingested and stored in memory, with optional filtering by tag or semantic search.
  • Arguments:
    • `tag` (string, optional): Filter sources by tag in chunk metadata.
    • `query` (string, optional): Semantic search query to find relevant sources.
    • `top_k` (integer, optional, default 10): Number of top sources to return when using query.
• get-context: Retrieve relevant content chunks (content only) for use as AI context, with filtering by tag, type, and semantic similarity.
  • Arguments:
    • `query` (string, optional): The semantic search query.
    • `tag` (string, optional): Filter results by a specific tag in chunk metadata.
    • `type` (string, optional): Filter results by chunk type (e.g., 'code', 'markdown').
    • `top_k` (integer, optional, default 5): The number of top relevant chunks to retrieve.
• update-chunk-metadata: Update the metadata field for a chunk by id
  • Arguments: `id` (integer), `metadata` (object)
• tag-chunks-by-source: Adds specified tags to the metadata of all chunks associated with a given source (URL or file path). Merges with existing tags.
  • Arguments: `source` (string), `tags` (list of strings)
• list-notes: List all currently stored notes and their content.

Chunking and Code Extraction

• Markdown, Python, OpenAPI, and HTML files are split into logical chunks for efficient retrieval and search.
• The markdown chunker uses mistune 3.x's AST API and regex to robustly split content by code blocks and narrative, preserving all original formatting.
• Both code blocks and markdown/narrative content are preserved as separate chunks.
• The HTML chunker uses the `readability-lxml` library to extract main content first, then extracts block code snippets from `` tags as dedicated "code" chunks. Inline `` content remains part of the narrative chunks.

Semantic Search

• The `search-chunks` tool performs vector-based semantic search over all ingested content, returning the most relevant chunks for a given query.
• Supports optional `type` and `tag` arguments to filter results by chunk type (e.g., `code`, `html`, `markdown`) and/or by tag in chunk metadata, before semantic ranking.
• This enables highly targeted retrieval, such as "all code chunks tagged with 'langfuse' relevant to 'cost and usage'".

Metadata Management

• Chunks include a `metadata` field for categorization and tagging.
• The `update-chunk-metadata` tool allows updating metadata for any chunk by its id.
• The `tag-chunks-by-source` tool allows adding tags to all chunks from a specific source in one operation. Tagging merges new tags with existing ones, preserving previous tags.

Configuration

The server requires the following environment variables (can be set in a .env file):

Ollama Configuration

• OLLAMA_HOST: Hostname for Ollama API (default: localhost)
• OLLAMA_PORT: Port for Ollama API (default: 11434)
• RAG_AGENT: Ollama model to use for RAG responses (default: llama3)
• OLLAMA_MODEL: Ollama model to use for embeddings (default: nomic-embed-text-v2-moe)

Database Configuration

• HOST: PostgreSQL database host (default: localhost)
• DB_PORT: PostgreSQL database port (default: 5432)
• DB_NAME: PostgreSQL database name (default: doclibdb)
• DB_USER: PostgreSQL database user (default: doclibdb_user)
• DB_PASSWORD: PostgreSQL database password (default: doclibdb_password)

Reranker Configuration

• RERANKER_MODEL_PATH: Path to the reranker model (default: /srv/samba/fileshare2/AI/models/bge-reranker-v2-m3)
• RERANKER_USE_FP16: Whether to use FP16 for reranker (default: True)

Quickstart

Install

Claude Desktop

On MacOS: `~/Library/Application\ Support/Claude/claude_desktop_config.json`

On Windows: `%APPDATA%/Claude/claude_desktop_config.json`

Development/Unpublished Servers Configuration

"mcpServers": {
    "doc-lib-mcp": {
      "command": "uv",
      "args": [
        "--directory",
        "/home/administrator/python-share/doc-lib-mcp",
        "run",
        "doc-lib-mcp"
      ]
    }
  }

Published Servers Configuration

"mcpServers": {
    "doc-lib-mcp": {
      "command": "uvx",
      "args": [
        "doc-lib-mcp"
      ]
    }
  }

Development

Building and Publishing

To prepare the package for distribution:

1. Sync dependencies and update lockfile:

uv sync

2. Build package distributions:

uv build

This will create source and wheel distributions in the `dist/` directory.

3. Publish to PyPI:

uv publish

Note: You'll need to set PyPI credentials via environment variables or command flags:

• Token: `--token` or `UV_PUBLISH_TOKEN`
• Or username/password: `--username`/`UV_PUBLISH_USERNAME` and `--password`/`UV_PUBLISH_PASSWORD`

Debugging

Since MCP servers run over stdio, debugging can be challenging. For the best debugging

experience, we strongly recommend using the MCP Inspector.

You can launch the MCP Inspector via `npm` with this command:

npx @modelcontextprotocol/inspector uv --directory /home/administrator/python-share/doc-lib-mcp run doc-lib-mcp

Upon launching, the Inspector will display a URL that you can access in your browser to begin debugging.