Chomper

License: MIT Python 3.10+ MCP

Chomp through any document. An MCP server that parses 36+ file formats for AI systems like Claude.

Features

• 15+ Format Categories: PDF, DOCX, PPTX, Excel, CSV, HTML, Markdown, Text, Code (10+ languages), JSON, YAML, XML, Email (EML/MSG), EPUB, RTF
• Smart Token Management: Summary mode by default (5000 chars), pagination for large documents
• TOON Output Format: Token-Optimized Object Notation reduces token usage by ~40%
• Semantic Chunking: Embedding-based chunking using sentence-transformers for better RAG retrieval
• Image Extraction: PDF images returned as ImageContent for direct AI analysis
• MCP Prompts: Built-in document analysis prompts (summarize, extract entities, Q&A, etc.)
• Rich Metadata: Author, title, pages, word count, reading time, complexity scores
• Batch Processing: Parse multiple documents in a single request

Quick Start

Installation

# Clone the repository
git clone https://github.com/IcHiGo-KuRoSaKiI/Chomper.git
cd chomper

# Create virtual environment and install
python -m venv venv
source venv/bin/activate  # or `venv\Scripts\activate` on Windows
pip install -e .

Running the Server

# Direct execution
python server.py

# Or via the installed command
chomper

Configure in Claude Code

claude mcp add -s user chomper -- /path/to/chomper/venv/bin/python /path/to/chomper/server.py

Configure in Claude Desktop

Add to your Claude Desktop configuration (~/Library/Application Support/Claude/claude_desktop_config.json on macOS):

{
  "mcpServers": {
    "chomper": {
      "command": "/path/to/chomper/venv/bin/python",
      "args": ["/path/to/chomper/server.py"]
    }
  }
}

Python Library

Chomper can be used as a standalone Python library for document parsing:

import chomper

# Parse a document
result = chomper.parse("/path/to/document.pdf")
print(result.text)
print(result.metadata)
print(f"Words: {result.word_count}, Format: {result.format}")

# Parse from base64 (cloud storage, APIs, databases)
import base64
with open("doc.pdf", "rb") as f:
    content = base64.b64encode(f.read()).decode()

result = chomper.parse_bytes(content, "doc.pdf")

# Quick metadata extraction
meta = chomper.extract_metadata("/path/to/report.pdf")
print(f"Author: {meta.author}, Pages: {meta.page_count}")

# Chunk for RAG/embeddings
chunks = chomper.chunk("/path/to/doc.pdf", strategy="semantic")
for chunk in chunks:
    print(f"Chunk {chunk.chunk_id}: {chunk.word_count} words")
    print(f"Keywords: {chunk.keywords}")

# Check format support
if chomper.is_supported("report.pdf"):
    result = chomper.parse("report.pdf")

# List all formats
formats = chomper.list_formats()
for ext, info in formats.items():
    if info["available"]:
        print(f"{ext}: {info['description']}")

API Reference

Function	Description
chomper.parse(file_path)	Parse document, returns ParseResult
chomper.parse_bytes(content, filename)	Parse from bytes/base64
chomper.chunk(file_path, strategy)	Split into chunks for RAG
chomper.extract_metadata(file_path)	Quick metadata extraction
chomper.list_formats()	List supported formats
chomper.is_supported(file_path)	Check if format supported

Result Objects

# ParseResult
result.text           # Extracted text content
result.metadata       # Document metadata dict
result.format         # File format (pdf, docx, etc.)
result.word_count     # Total word count
result.char_count     # Total character count

# ChunkResult (from chomper.chunk())
chunk.text            # Chunk text
chunk.chunk_id        # Chunk index (0-based)
chunk.word_count      # Words in chunk
chunk.keywords        # Extracted keywords
chunk.section_name    # Detected section name

# MetadataResult (from chomper.extract_metadata())
meta.filename         # Base filename
meta.format           # File format
meta.file_size        # Size in bytes
meta.author           # Author (if available)
meta.title            # Title (if available)
meta.page_count       # Pages (if applicable)

Command-Line Interface

Parse documents directly from the command line:

# Parse and print text
chomper-parse document.pdf

# Output as JSON
chomper-parse report.docx --json

# Output in different formats (csv, markdown, xml)
chomper-parse report.pdf --format markdown
chomper-parse data.xlsx --format csv

# Show metadata only
chomper-parse data.xlsx --metadata

# Split into chunks
chomper-parse book.pdf --chunk --strategy semantic

# Save to file
chomper-parse document.pdf -o output.txt

# List supported formats
chomper-parse --formats

# Quiet mode (no progress messages)
chomper-parse document.pdf -q

Output Formats

# Plain text (default)
chomper-parse document.pdf

# JSON output
chomper-parse document.pdf --format json
chomper-parse document.pdf --json  # shortcut

# CSV output
chomper-parse document.pdf --format csv

# Markdown output
chomper-parse document.pdf --format markdown

# XML output
chomper-parse document.pdf --format xml

# Custom Jinja2 template
chomper-parse document.pdf --format template --template my_template.j2

Watch Mode

Monitor a directory for new/changed files and auto-parse them:

# Watch a directory
chomper-parse --watch ./documents

# Watch with JSON output saved to files
chomper-parse --watch ./inbox --format json --output-dir ./parsed

# Watch only PDFs, check every 5 seconds
chomper-parse --watch ./docs --pattern "*.pdf" --interval 5

# Watch recursively (including subdirectories)
chomper-parse --watch ./project --recursive

# Watch with metadata only
chomper-parse --watch ./docs --metadata --format json

Interactive Mode

Start an interactive shell for parsing multiple documents:

$ chomper-parse -i
Chomper Interactive Mode
Type 'help' for commands, 'exit' to quit.

chomper> parse ~/Documents/report.pdf
[Document content displayed...]

chomper> set format json
Output format set to: json

chomper> metadata ~/Documents/report.pdf
{
  "filename": "report.pdf",
  "format": "pdf",
  "page_count": 5
}

chomper> history
Files parsed this session:
  1. /Users/me/Documents/report.pdf

chomper> help
[Shows all available commands]

chomper> exit

Interactive Commands:

Command	Description
parse <file>	Parse a document
metadata <file>	Show metadata only
chunk <file>	Split into chunks
formats	List supported formats
set format <name>	Set output format
set json on/off	Toggle JSON mode
set max-chars N	Limit output
history	Show parsed files
status	Show current settings
help	Show all commands
exit	Exit interactive mode

CLI Options

Option	Description
-f, --format	Output format: text, json, csv, markdown, xml, template
--template	Jinja2 template file (with --format template)
--json	Shortcut for --format json
--metadata	Show metadata only
--chunk	Split into chunks
--strategy	Chunking: auto, semantic, fixed
--chunk-size	Words per chunk (default: 1000)
--max-chars	Limit output characters
-o, --output	Save to file
-i, --interactive	Start interactive mode
-w, --watch	Watch directory for changes
--interval	Watch interval in seconds (default: 2)
--output-dir	Save watch output to directory
--pattern	File pattern for watch mode
--recursive	Watch subdirectories
--formats	List supported formats
-q, --quiet	Suppress progress messages

MCP Tools

The following tools are available via the MCP server:

Available Tools

1. parse_document

Parse a document and extract text, metadata, and images. Returns summary by default (first 5000 chars) to stay within token limits.

Parameters:

Name	Type	Default	Description
file_path	string	required	Absolute path to the document
full_text	boolean	false	Return complete text (may exceed token limits)
include_images	boolean	false	Include images as ImageContent
output_format	string	"json"	Output format: "json" or "toon" (token-optimized)

Response:

• TextContent[0]: Plain extracted text (no JSON wrapping)
• TextContent[1]: Metadata as JSON (includes continuation hint if truncated)
• ImageContent[]: Images if include_images=true

Example:

parse_document(file_path: "/path/to/doc.pdf")
→ Returns first 5000 chars + metadata with hint to fetch more

parse_document(file_path: "/path/to/doc.pdf", output_format: "toon")
→ Returns in TOON format (~40% fewer tokens)

2. parse_document_bytes

Parse a document from base64-encoded content. Perfect for documents from cloud storage (S3, Azure Blob), API responses, database BLOBs, or in-memory documents.

Parameters:

Name	Type	Default	Description
content_base64	string	required	Base64-encoded file content
filename	string	required	Filename with extension (e.g., "report.pdf") for format detection
full_text	boolean	false	Return complete text
include_images	boolean	false	Include images as ImageContent
output_format	string	"json"	Output format: "json" or "toon"

Example:

import base64

# Read file and encode to base64
with open("document.pdf", "rb") as f:
    content = base64.b64encode(f.read()).decode()

# Send via MCP
parse_document_bytes(
    content_base64=content,
    filename="document.pdf"
)
→ Returns extracted text + metadata (same as parse_document)

Use Cases:

• Documents fetched from cloud storage (S3, Azure Blob, GCS)
• Files received from API responses
• Documents stored as BLOBs in databases
• In-memory document processing without disk I/O

4. get_document_chunk

Get a specific portion of document text. Use for paginated retrieval of large documents.

Parameters:

Name	Type	Default	Description
file_path	string	required	Absolute path to the document
offset	integer	0	Character offset to start from
limit	integer	5000	Maximum characters to return
output_format	string	"json"	Output format: "json" or "toon"

Example workflow:

1. parse_document(file_path: "doc.pdf")
   → Returns chars 0-5000, hint: "use get_document_chunk(offset=5000)"

2. get_document_chunk(file_path: "doc.pdf", offset: 5000)
   → Returns chars 5000-10000

3. get_document_chunk(file_path: "doc.pdf", offset: 10000)
   → Returns chars 10000-15000, etc.

5. get_document_images

Retrieve images from a document on-demand. Returns images as ImageContent objects.

Parameters:

Name	Type	Default	Description
file_path	string	required	Absolute path to the document
page	integer	all	Specific page number (1-indexed)
max_images	integer	5	Maximum images to return

Example:

get_document_images(file_path: "doc.pdf", page: 1, max_images: 3)
→ Returns first 3 images from page 1 as ImageContent

6. parse_document_chunked

Parse a document into semantic chunks with configurable size and overlap. Ideal for RAG systems.

Parameters:

Name	Type	Default	Description
file_path	string	required	Absolute path to the document
chunk_size	integer	1000	Target words per chunk
overlap	integer	100	Words to overlap between chunks
chunking_strategy	string	"auto"	Strategy: "auto", "semantic", "fixed", "recursive"
embedding_model	string	"fast"	For semantic: "fast" (~80MB) or "balanced" (~420MB)
output_format	string	"json"	Output format: "json" or "toon"

Chunking Strategies:

• auto: Format-aware chunking (uses specialized chunker per file type)
• semantic: Embedding-based chunking using sentence-transformers (best for RAG)
• fixed: Simple character count splitting
• recursive: Paragraph/sentence boundary splitting

Response (JSON):

{
  "success": true,
  "total_chunks": 25,
  "chunking_strategy": "semantic",
  "embedding_model": "fast",
  "chunks": [
    {
      "chunk_id": 0,
      "text": "Chunk content...",
      "word_count": 250,
      "keywords": ["key", "terms"],
      "section_name": "Introduction",
      "metadata": {
        "chunk_strategy": "semantic",
        "breakpoint_strategy": "percentile"
      }
    }
  ],
  "statistics": {
    "total_words": 6000,
    "average_chunk_words": 240
  }
}

7. extract_metadata

Quick metadata extraction without full document processing.

Parameters:

Name	Type	Default	Description
file_path	string	required	Absolute path to the document
output_format	string	"json"	Output format: "json" or "toon"

Response (JSON):

{
  "success": true,
  "metadata": {
    "author": "John Doe",
    "title": "Document Title",
    "page_count": 10
  },
  "document_info": {
    "text_length": 35000,
    "image_count": 5
  }
}

8. list_supported_formats

List all supported document formats with availability status.

9. batch_parse

Parse multiple documents in a single request.

Parameters:

Name	Type	Default	Description
file_paths	string[]	required	Array of file paths
include_images	boolean	false	Include images
continue_on_error	boolean	true	Continue if a file fails

MCP Prompts

The server exposes 5 document analysis prompts that can be used with Claude:

Prompt	Description	Arguments
summarize-document	Generate comprehensive document summary	file_path, length (short/medium/long)
extract-key-points	Extract main takeaways and key points	file_path, max_points
explain-document	Explain document for different audiences	file_path, audience (child/general/expert)
extract-entities	Extract named entities (people, orgs, locations)	file_path, entity_types
document-qa	Set up Q&A context for document	file_path

Usage in Claude:

Use the summarize-document prompt with file_path="/path/to/doc.pdf"

TOON Format (Token-Optimized Output)

TOON format reduces token usage by ~40% compared to JSON, ideal for LLM contexts:

d:report.pdf|t:pdf|w:5000|c:25000|n:10
m:author=John Doe,title=Annual Report
---
0|0-2500|text|Introduction
The document begins with an overview...
k:overview,introduction,summary
---
1|2500-5000|text|Methodology
The methodology section describes...
k:methodology,approach,methods

Enable with: output_format: "toon" on any tool.

Supported Formats

Category	Extensions	Description
Documents	.pdf, .docx, .doc, .pptx, .ppt	Office documents with full structure
Spreadsheets	.xlsx, .xlsm, .xltx, .xltm, .csv, .tsv	Tables with type inference
Web	.html, .htm, .md, .markdown	Semantic structure preservation
Text	.txt, .text, .log	Plain text with paragraph detection
Code	.py, .js, .ts, .jsx, .tsx, .java, .cpp, .c, .go, .rs	Language-aware parsing
Data	.json, .yaml, .yml, .xml	Structured data with schema detection
Email	.eml, .msg	Email with headers, body, attachments
E-books	.epub	Chapter extraction with TOC
Rich Text	.rtf	Rich Text Format documents

Total: 36 file extensions supported

Recommended Usage Pattern

For best results with AI systems that have token limits:

# 1. Start with summary (default behavior)
parse_document(file_path: "large_doc.pdf")

# 2. If you need more content, paginate
get_document_chunk(file_path: "large_doc.pdf", offset: 5000)
get_document_chunk(file_path: "large_doc.pdf", offset: 10000)

# 3. Fetch images separately when needed
get_document_images(file_path: "large_doc.pdf", max_images: 3)

# 4. For RAG pipelines, use semantic chunking
parse_document_chunked(file_path: "doc.pdf", chunking_strategy: "semantic")

Avoid:

# DON'T use full_text=true for large documents - will exceed token limits!
parse_document(file_path: "large_doc.pdf", full_text: true)  # Bad

Architecture

The server wraps a 4-layer document processing pipeline:

┌─────────────┐    ┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│  Extractors │ -> │  Chunkers   │ -> │  Enrichers  │ -> │ Formatters  │
│ (Layer 1)   │    │ (Layer 2)   │    │ (Layer 3)   │    │ (Layer 4)   │
└─────────────┘    └─────────────┘    └─────────────┘    └─────────────┘
     │                   │                  │                   │
     v                   v                  v                   v
 Raw text +         Semantic           Keywords +         JSON/TOON
 Structure          Chunks             Metadata            Output

Extractors: Format-specific text and metadata extraction Chunkers: Auto, semantic (embeddings), fixed, recursive strategies Enrichers: Keywords, sections, titles, complexity scores Formatters: JSON (default) or TOON (token-optimized)

Dependencies

Core (always available):

• mcp>=1.0.0 - Model Context Protocol
• Code, Text, Markdown extractors (no heavy dependencies)

Optional (for additional formats):

• PDF: pymupdf, pymupdf4llm, pillow
• Office: python-docx, python-pptx, openpyxl
• Web: beautifulsoup4, lxml, trafilatura
• Data: pyyaml (YAML), lxml (XML)
• Email: extract-msg (MSG files)
• E-books: ebooklib (EPUB)
• Rich Text: striprtf (RTF)
• Semantic Chunking: sentence-transformers

Install all dependencies:

pip install -r requirements.txt

Error Handling

All responses include appropriate error information on failure:

{
  "success": false,
  "error": "File not found: /path/to/missing.pdf",
  "error_type": "ValueError"
}

Development

# Install dev dependencies
pip install -e ".[dev]"

# Run tests
pytest
python src/tests/test_lightweight.py

# Format code
black .

# Lint
ruff check .

Comparison with Other Tools

Feature	Chomper	LlamaParse	Docling	Unstructured
MCP Native	Yes	No	No	No
Format Count	36	~15	~10	~20
Token Optimization	TOON (~40% savings)	No	No	No
Semantic Chunking	Built-in	Separate	Separate	Separate
MCP Prompts	5 built-in	No	No	No
Complex Tables	Good (pymupdf4llm)	Excellent	Excellent (AI)	Average
Cloud Required	No (local)	Yes	No	Optional
Cost	Free	Paid	Free	Freemium

Contributing

Contributions are welcome! Please read CONTRIBUTING.md for guidelines.

Quick Start for Contributors

# Fork and clone
git clone https://github.com/YOUR_USERNAME/chomper.git
cd chomper

# Setup dev environment
python -m venv venv
source venv/bin/activate
pip install -e ".[dev]"

# Run tests
pytest

# Format code
black .
ruff check .

License

MIT License - see LICENSE for details.

---

Built with love by @IcHiGo-KuRoSaKiI