LLM Chat Server

MCP server for interacting with LLM models configured in Continue.dev's config.yaml.

What it does

Exposes 4 tools that let the agent inspect your configured models and send chat requests to them:

• llm_chat_list_models — list all models in your config with their names and ids
• llm_chat_get_model_params — inspect parameters (temperature, topP, etc.) for any model
• llm_chat_get_model_prompt — read the system prompt configured for a model
• llm_chat_send_request — chat with a model, optionally overriding parameters or loading prompts from files

Installation

pip install -e .

Or run directly from source without install.

CLI Usage

python -m llm_chat_mcp --default-model "GLM-5.2-FP8"
python -m llm_chat_mcp --config /path/to/config.yaml --default-model "ModelName"
python -m llm_chat_mcp --help

Configuration

• Config is re-read on every request — no restart needed
• Auth uses apiKey from each model entry in config.yaml
• If no model is specified (neither in CLI nor tool call), an error tells you how to set one

llm_chat_send_request parameters

The main tool. Sends a chat completion request to an LLM and returns the response.

Input parameters

Response structure

Always returned as JSON:

{
  "content": "<preview, full content, or empty>",
  "truncated": true,
  "metadata": {
    "model_name": "...",
    "model": "...",
    "elapsed_seconds": 1.23,
    "request_sent": {...},
    "response_headers": {...},
    "response_chars": 1234,
    "output_file": "...",
    "auto_output_file": "...",
    "created_dirs": [...],
    "appended_line_start": 201,
    "appended_line_end": 250
  }
}

Output strategy

The tool chooses one of three strategies based on parameters and response size:

1. Explicit file (output_file_path set): full response written to the file. If append=true, the response is appended and appended_line_start/appended_line_end (1-based, inclusive) are returned so the caller can read only the appended slice via extract_lines.
2. Auto file (no output_file_path, auto_file_threshold > 0, response_chars > threshold): full response written to an auto-generated file in --auto-output-dir (or OS temp). Filename format: llm_output_<YYYYMMDD_HHMMSS>_<6-char-uuid>.json.
3. Inline only (no file written): full content returned in the content field.

Inline preview

When a file is written (explicit or auto), the content field contains a preview of the response:

• If inline_preview_chars > 0 and len(content) > inline_preview_chars: truncated preview with suffix [truncated, full response in <file_path>].
• If inline_preview_chars > 0 and len(content) <= inline_preview_chars: full content (fits in preview).
• If inline_preview_chars == 0: empty content (file has the full response).

When no file is written, the full content is returned inline regardless of inline_preview_chars — this prevents data loss.

Continue.dev Integration

Add to .continue/mcpServers/llm-chat.yaml:

name: LLM Chat MCP server
version: 0.2.0
schema: v1
mcpServers:
  - name: LLM Chat MCP server
    command: python
    args:
      - "-m"
      - "llm_chat_mcp"
      - "--default-model"
      - "GLM-5.2-FP8"
      - "--timeout"
      - "570"
      - "--relative_paths_base"
      - "/path/to/your/workspace"
      - "--auto-output-dir"
      - "/path/to/your/workspace/.continue/skills/large-tasks/tmp-outputs"
    env:
      PYTHONPATH: "/path/to/llm-chat-mcp"

Then reload Continue.dev.

Project Structure

llm-chat-mcp/
├── pyproject.toml          # Dependencies: mcp, pyyaml, httpx
├── README.md               # This file
├── llm_chat_mcp/
│   ├── __init__.py
│   ├── __main__.py         # CLI entry point + tool registration
│   ├── config.py           # Config loading, model resolution
│   └── api.py              # API client, error handling
└── tests/
    └── test_output_strategies.py  # Tests for append, inline_preview, auto_file

Testing

python tests/test_output_strategies.py

Tests mock the API call and verify the file-writing and response assembly logic. Covers all combinations of output_file_path, append, inline_preview_chars, and auto_file_threshold.