Document Understanding MCP Server

> ⚠️ WARNING: This server is currently in alpha stage and is intended for local development and testing only. It should not be deployed in production environments or exposed to untrusted networks. Please review SECURITY.md for important security considerations before running this server.

MCP server providing tools to extract text, metadata, layout, and search documents (primarily PDFs). This server implements the Model Context Protocol (MCP) specification, allowing AI models to interact with PDF documents through a standardized interface.

Overview

The Document Understanding MCP Server provides a set of tools for extracting information from PDF documents, including:

• Text content extraction (with OCR fallback for scanned documents)
• Metadata extraction (author, title, creation date, etc.)
• Layout information extraction (text blocks, images, drawings)
• Table extraction
• Image extraction
• Document outline/bookmarks extraction
• Text search functionality
• Language detection

These tools can be used by AI models to analyze and understand PDF documents, enabling more sophisticated document processing workflows.

> Future Plans: We're working on expanding support to multiple document types beyond PDF. See docs/plans/README.md for details on upcoming architectural changes.

Setup and Installation

1. Clone the repository:

git clone 
    cd document-understanding-mcp-server

2. Install Python: Ensure you have Python 3.11+ installed.

3. Install `uv`: It's recommended to use `uv` for environment management.

pip install uv

4. Create and Sync Virtual Environment:

uv venv # Create .venv
    # Install core dependencies
    uv pip sync pyproject.toml
    # For development/testing, install optional extras:
    uv pip install -e '.[dev,test]'

5. (Required for Table Extraction) Install Java: The `extract_tables` tool relies on `tabula-py`, which requires a Java Runtime Environment (JRE). Install a JRE (e.g., OpenJDK) suitable for your system and ensure `java` is accessible in your system's PATH.

6. (Required for OCR Fallback) Install Tesseract: The OCR fallback in `extract_pdf_contents` relies on `pytesseract`, which requires the Tesseract OCR engine itself. Install Tesseract for your OS (see Tesseract Installation) and ensure the `tesseract` command is in your system's PATH.

7. Configure Base Path (Mandatory unless overridden): Define the `DOCUMENT_UNDERSTANDING_BASE_PATH` environment variable. This is the only directory the server will read PDFs from by default. Example:

export DOCUMENT_UNDERSTANDING_BASE_PATH=/path/to/your/pdf_working_directory

Configuration

• **`DOCUMENT_UNDERSTANDING_BASE_PATH` (Environment Variable, Required*):** Specifies the absolute path to the directory the server is allowed to access for reading PDFs. The server will refuse to start if this is not set, unless `--allow-any-path` is used.
• `DOCUMENT_UNDERSTANDING_LOG_LEVEL` (Environment Variable): Sets the minimum log level for console output (e.g., `DEBUG`, `INFO`, `WARNING`). Default=`INFO`.
• `DOCUMENT_UNDERSTANDING_LOG_FORMAT` (Environment Variable): Sets console log format (`plain` or `json`). Default=`plain`.
• `DOCUMENT_UNDERSTANDING_LOG_FILE` (Environment Variable): If set, specifies a path to write structured JSON logs (e.g., `.tmp/logs/server.log`). Directory will be created. Default=Disabled.
• `DOCUMENT_UNDERSTANDING_LOG_FILE_LEVEL` (Environment Variable): Minimum level for file logging (e.g., `DEBUG`, `INFO`). Default=`INFO`.
• `DOCUMENT_UNDERSTANDING_DEFAULT_LANG` (Environment Variable): Sets the default OCR language (e.g., `eng`, `fra`). Defaults to `eng` if not set.
• `ENABLE_SAVE_IMAGES_TO_FILES` (Environment Variable): When set to `true`, enables saving extracted images to files when using the `extract_images` tool with the `output_directory` parameter. Default=`false`.
• `SAFE_OUTPUT_DIRECTORIES` (Environment Variable): Colon-separated list of directories where images can be saved when `ENABLE_SAVE_IMAGES_TO_FILES=true` but `ALLOW_ANY_PATH=false`. For example: `/tmp:/var/output:/data/images`.
• Capability Override / Enable Flags (Command-line):

Capability / Dependency Matrix

The availability or full functionality of certain tools depends on external dependencies or flags.

*\**Note:* If Tesseract OCR is unavailable (and `--allow-no-tesseract` is used), OCR fallback is disabled.*

Tools

The server implements the following tools. The list presented to the LLM via the `list_tools` call is dynamic and depends on detected capabilities (e.g., Java availability) and startup flags.

See `src/document_understanding/models.py` for response schemas (e.g., `TextContentResponse`, `MetadataResponse`, etc.). Error responses follow the `ErrorResponse` schema.

*Guidance:* The `pdf_path` argument in all tools must refer to a file path accessible within the server's environment.

1. `extract_pdf_contents`

• Description: Extracts text content from specified pages of a local PDF file. Uses direct text extraction with OCR fallback for scanned or image-heavy pages.
• Arguments:
• Returns: `TextContentResponse` JSON.
• Guidance: Best for getting the raw text of pages. Handles OCR automatically if needed. Use `pages` for specific sections. If dealing with non-English scanned text, set `ocr_language`.

2. `extract_pdf_metadata`

• Description: Extracts metadata (author, title, dates) and checks for images/drawings in a PDF.
• Arguments:
• Returns: `MetadataResponse` JSON.
• Guidance: Use this first to get an overview of the PDF structure and properties (page count, title, author, etc.) and a hint about whether images/drawings are present.

3. `search_pdf_text`

• Description: Searches for exact text occurrences within specified pages and returns bounding boxes.
• Arguments:
• Returns: `SearchResponse` JSON containing a list of matches with page number and rectangle coordinates.
• Guidance: Useful for finding specific terms or phrases and their location.

4. `extract_pdf_layout`

• Description: Extracts detailed layout info: text blocks, drawings, image placements (with coordinates).
• Arguments:
• Returns: `LayoutResponse` JSON containing layout details per page.
• Guidance: Use when spatial relationships are important or to understand page structure. Provides coordinates for text, images, and vector drawings. Can generate large responses for complex pages.

5. `extract-images`

• Description: Extracts info about images (raster images, some other objects like Form XObjects).
• Arguments:
• Returns: `ImageExtractionResponse` JSON.
• Guidance: Returns image dimensions, page number, and internal reference (`xref`). Bounding box (`bbox`) is optional and may be missing if detection fails (e.g., for Form XObjects). Use `include_data=True` cautiously as it can return very large responses. When `output_directory` is specified and `ENABLE_SAVE_IMAGES_TO_FILES=true`, images will be saved to disk and the response will include file paths. Use `save_without_returning_data=True` to save images to files without including the potentially large base64 data in the response.

6. `extract_tables`

• Description: Extracts tables from specified PDF pages into lists of lists.
• Arguments:
• Returns: `TableExtractionResponse` JSON. (Note: Current implementation returns list of dicts, see docs/KNOWN_ISSUES.md).
• Guidance: Best effort table detection using `tabula-py`. Results depend heavily on table formatting (lines, spacing). May be slow. Note: This tool requires a Java runtime. If Java is not detected at server startup and the `--allow-no-java` flag was used, this tool will be unavailable.

7. `detect-language`

• Description: Detects the language(s) of text content sampled from specified pages (defaults to page 1).
• Arguments:
• Returns: `LanguageDetectionResponse` JSON.
• Guidance: Useful for determining the primary language before processing or translation.

8. `extract_pdf_outline`

• Description: Extracts the document outline (bookmarks/table of contents) from a PDF.
• Arguments:
• Returns: `OutlineExtractionResponse` JSON containing the hierarchical outline structure.
• Guidance: Use to extract the document's table of contents or bookmark structure. Returns an empty list if the document has no outline.

9. `get_pdf_working_directory`

• Description: Returns the designated directory path where PDF files should be placed for processing.
• Arguments: None.
• Returns: JSON with `status`, `working_directory` (string or null), and optional `message`.
• Guidance: Call this tool to find out where to upload/place PDF files before calling other tools that require a `pdf_path`. If the server was started with `--allow-any-path`, `working_directory` will be null.

Testing

The test suite uses dynamically generated PDFs for all tests. No static PDF files are stored in the repository. The test fixtures in `tests/conftest.py` generate all necessary test PDFs at runtime, including:

• Simple text documents
• Documents with images
• Documents with drawings
• Documents with tables
• Documents with outlines/bookmarks
• Empty documents
• Encrypted documents

This approach ensures that all tests can run without requiring external files and makes the test suite more portable and self-contained.

Running Tests

# Run all tests
python -m pytest

# Run specific test categories
python -m pytest tests/unit
python -m pytest tests/integration

# Run tests with coverage
python -m pytest --cov=src

Test Architecture

The test suite includes several types of tests:

• Unit Tests: Test individual components in isolation
• Integration Tests: Test the interaction between components
• End-to-End Tests: Test the entire system from the user's perspective

The end-to-end tests use the MCP command-line interface to interact with the server, simulating real-world usage. This approach ensures that the server works correctly in various scenarios and with different types of PDFs.

Running the Server / Invocation

This server runs using the standard MCP stdio communication. There are multiple ways to run the server:

Method 1: Using the Provided Executable (Recommended)

# Set required base path (if not using sandbox mode)
export DOCUMENT_UNDERSTANDING_BASE_PATH=/path/to/pdf/storage

# Run with default settings (sandbox mode)
bin/document-understanding-mcp-server

# Run with sandbox mode disabled (less restricted)
DOCUMENT_UNDERSTANDING_SANDBOX=false bin/document-understanding-mcp-server

# Run with additional arguments
bin/document-understanding-mcp-server --enable-experimental --port 8000

The `document-understanding-mcp-server` executable automatically sets up a secure environment with:

• User-specific isolated directories in sandbox mode (default)
• Appropriate permissions based on security context
• Intelligent argument handling based on environment

Method 2: Using Installed Package Entry Point

When the package is installed via pip, you can use the entry point directly:

# Run with default settings
document-understanding-mcp-server

# Run with custom arguments
document-understanding-mcp-server --allow-any-path --enable-experimental

Method 3: Using `standalone_server.py` Directly

# Activate environment
source .venv/bin/activate

# Set required base path
export DOCUMENT_UNDERSTANDING_BASE_PATH=/path/to/pdf/storage

# Run directly (will fail if Java/Tesseract missing and flags not used)
python standalone_server.py

# Run allowing missing Java/Tesseract but restricting path
python standalone_server.py --allow-no-java --allow-no-tesseract

# Run allowing ANY path access (UNSAFE)
# DOCUMENT_UNDERSTANDING_BASE_PATH is ignored here
python standalone_server.py --allow-any-path

# Enable experimental features
python standalone_server.py --enable-experimental

Sandbox Mode

The server supports a secure sandbox mode (enabled by default) that:

1. Creates isolated user-specific directories for file output

2. Sets stricter file permissions (700 - user access only)

3. Disables the `--allow-any-path` flag for better security

To disable sandbox mode (not recommended for production):

export DOCUMENT_UNDERSTANDING_SANDBOX=false

Invocation Command (Example `mcp.json`)

Example 1: Basic Configuration

{
  "mcpServers": {
    "document-understanding": {
      "command": "/path/to/document-understanding-mcp-server",
      "args": [
        "--enable-experimental"
      ]
    }
  }
}

Example 2: Using Python Module Directly

{
  "mcpServers": {
    "document-understanding": {
      "command": "/path/to/your/.venv/bin/python",
      "args": [
        "-m", "document_understanding.cli"
      ],
      "env": {
        "DOCUMENT_UNDERSTANDING_BASE_PATH": "/path/to/pdf/storage"
      }
    }
  }
}

LLM Usage Advice

• File Paths: Before processing a PDF, use `get_pdf_working_directory` to find the allowed directory. Ensure any `pdf_path` arguments you provide refer to files within that directory (or subdirectories). Path traversal attempts (`../`) or absolute paths outside the working directory will be rejected unless the server was unsafely started with `--allow-any-path`.
• Passwords: Currently, the server cannot handle password-protected PDFs for most tools (except table extraction). Processing encrypted files will likely fail. See docs/KNOWN_ISSUES.md.
• Tool Availability: The list of available tools might change depending on the server's environment (e.g., if Java is missing and `--allow-no-java` was used, `extract_tables` will not be listed). Rely on the response from `list_tools` to know what is currently usable.
• Use specific tools: Prefer `search_pdf_text` over extracting all content if you only need specific occurrences. Use `extract_pdf_metadata` if only metadata is needed.
• Handle JSON: Expect JSON responses. Parse the `text` field of the `ToolResponseContent` as JSON.
• Check `status`: Always check the `status` field in the JSON response. If it's `"error"`, use the `message` and `error_code` fields for diagnostics (Note: errors are typically raised as exceptions by the server, the MCP client translates these).
• Layout Information: The `extract_pdf_layout` tool provides rich structural information (text coordinates, drawings, image locations). This can be used for complex analysis (e.g., with `find_nearby_content`) but generates large responses.
• Page Specification: Use the `pages` parameter effectively (e.g., "1-3, 5, -1") to limit processing when needed.
• OCR Language: If processing non-English documents, specify the language(s) using `ocr_language` (e.g., `"fra+eng"` for French and English) for better accuracy.
• Error Handling: Be prepared for `FileNotFoundError` if the `pdf_path` is incorrect, or `ValueError` for invalid page specifications or processing issues. Encrypted files may cause various errors. The server raises exceptions which the MCP client should surface.

Complex Usage Examples

1. Extracting French Text from Scanned Pages 2 & 3:

2. Finding Text near an Image:

3. Extracting Specific Tables after Metadata Check:

Critical Use Pathways

1. Extract Specific Pages:

Known Issues and Limitations

For a list of current known issues and limitations, see docs/KNOWN_ISSUES.md.

For a list of previously resolved issues, see docs/SOLVED_ISSUES.md.

Future Plans

For information about planned enhancements and future development, see the plans directory.

Contributing

For information on how to contribute to this project, please see the CONTRIBUTING.md file.

License

This project is licensed under the terms specified in the LICENSE file.