Waifu Queue

This project implements an MCP (Model Context Protocol) server for a conversational AI "waifu" character, leveraging the OpenRouter API via a Redis queue for asynchronous processing. It utilizes the FastMCP library for simplified server setup and management.

Table of Contents

• Features
• Architecture
• Prerequisites
• Installation
• Configuration
• Running the Service
• MCP API
• Testing
• Troubleshooting
• Contributing
• License

Features

• Text generation via OpenRouter using model from ~/.model-openrouter or openrouter/free.
• Request queuing using Redis for handling concurrent requests asynchronously.
• MCP-compliant API using FastMCP.
• Job status tracking via MCP resources.
• Configuration via environment variables (.env file).
• API key loading:
  • OpenRouter: OPENROUTER_API_KEY or ~/.api-openrouter
• Model selection files in home directory:
  • ~/.model-openrouter for OpenRouter model name

Architecture

The project consists of several key components:

• main.py: The main entry point, initializing the FastMCP application and defining MCP tools/resources.
• respond.py: Contains the core text generation logic using the OpenRouter API.
• task_queue.py: Handles interactions with the Redis queue (using python-rq), enqueuing generation requests.
• utils.py: Contains utility functions, specifically call_predict_response which is executed by the worker to call the generation logic in respond.py.
• worker.py: A Redis worker (python-rq) that processes jobs from the queue, calling call_predict_response.
• config.py: Manages configuration using pydantic-settings.
• models.py: Defines Pydantic models for MCP request and response validation.

The flow of a request is as follows:

1. A client sends a request to the generate_text MCP tool (defined in main.py).
2. The tool enqueues the request (prompt) to a Redis queue (handled by task_queue.py).
3. A worker.py process picks up the job from the queue.
4. The worker executes the call_predict_response function (from utils.py).
5. call_predict_response calls the predict_response function (in respond.py), which interacts with the OpenRouter API.
6. The generated text (or an error message) is returned by predict_response and stored as the job result by RQ.
7. The client can retrieve the job status and result using the job://{job_id} MCP resource (defined in main.py).

graph LR
    subgraph Client
        A[User/Client] -->|1. Send Prompt via MCP Tool| B(mcp-waifu-queue: main.py)
    end
    subgraph mcp-waifu-queue Server
        B -->|2. Enqueue Job (prompt)| C[Redis Queue]
        B -->|7. Return Job ID| A
        D[RQ Worker (worker.py)] --|>| C
        D -->|3. Dequeue Job & Execute| E(utils.call_predict_response)
        E -->|4. Call Generation Logic| F(respond.predict_response)
        F -->|5. Call OpenRouter API| G[OpenRouter API]
        G -->|6. Return Response| F
        F --> E
        E -->|Update Job Result in Redis| C
        A -->|8. Check Status via MCP Resource| B
        B -->|9. Fetch Job Status/Result| C
        B -->|10. Return Status/Result| A
    end

Prerequisites

• Python 3.7+
• pip or uv (Python package installer)
• Redis server (installed and running)
• An OpenRouter API Key

You can find instructions for installing Redis on your system on the official Redis website: https://redis.io/docs/getting-started/ You can obtain an OpenRouter API key from: https://openrouter.ai/

Installation

1. Clone the repository:

   git clone <YOUR_REPOSITORY_URL>
   cd mcp-waifu-queue
   

2. Create and activate a virtual environment using uv:

   python -m uv venv .venv
   .venv/Scripts/python.exe -m ensurepip
   .venv/Scripts/python.exe -m pip install uv
   

3. Install dependencies:

   .venv/Scripts/python.exe -m uv pip install -r requirements.txt
   .venv/Scripts/python.exe -m uv pip install -r requirements-dev.txt
   

Configuration

1. Model Names via files in $HOME:

   • OpenRouter model file:

     echo "openrouter/free" > ~/.model-openrouter
     

2. API Keys: Preferred via environment variables with file fallback:

   • OpenRouter: OPENROUTER_API_KEY or ~/.api-openrouter

   echo "YOUR_API_KEY_HERE" > ~/.api-openrouter
   

   (Replace YOUR_API_KEY_HERE with your actual key)

3. Other Settings: Copy the .env.example file to .env:

   cp .env.example .env
   

4. Modify the .env file to set the remaining configuration values:

   • MAX_NEW_TOKENS: Maximum number of tokens for the response (default: 2048).
   • REDIS_URL: The URL of your Redis server (default: redis://localhost:6379).
   • FLASK_ENV, FLASK_APP: Optional, related to Flask if used elsewhere, not core to the MCP server/worker operation.

Running the Service

1. Ensure Redis is running. If you installed it locally, you might need to start the Redis server process (e.g., redis-server command, or via a service manager).

2. Start the RQ Worker: Open a terminal, activate your virtual environment (source .venv/bin/activate or similar), and run:

   python -m mcp_waifu_queue.worker
   

   This command starts the worker process, which will listen for jobs on the Redis queue defined in your .env file. Keep this terminal running.

3. Start the MCP Server: Open another terminal, activate the virtual environment, and run the MCP server using a tool like uvicorn (you might need to install it: pip install uvicorn or uv pip install uvicorn):

   uvicorn mcp_waifu_queue.main:app --reload --port 8000 # Example port
   

   Replace 8000 with your desired port. The --reload flag is useful for development.

   Alternatively, you can use the start-services.sh script (primarily designed for Linux/macOS environments) which attempts to start Redis (if not running) and the worker in the background:

   # Ensure the script is executable: chmod +x ./scripts/start-services.sh
   ./scripts/start-services.sh
   # Then start the MCP server manually as shown above.
   

MCP API

The server provides the following MCP-compliant endpoints:

Tools

• generate_text
  • Description: Sends a text generation request to the OpenRouter API via the background queue.
  • Input: {"prompt": "Your text prompt here"} (Type: GenerateTextRequest)
  • Output: {"job_id": "rq:job:..."} (A unique ID for the queued job)

Resources

• job://{job_id}
  • Description: Retrieves the status and result of a previously submitted job.
  • URI Parameter: job_id (The ID returned by the generate_text tool).
  • Output: {"status": "...", "result": "..."} (Type: JobStatusResponse)
    • status: The current state of the job (e.g., "queued", "started", "finished", "failed"). RQ uses slightly different terms internally ("started" vs "processing", "finished" vs "completed"). The resource maps these.
    • result: The generated text if the job status is "completed", otherwise null. If the job failed, the result might be null or contain error information depending on RQ's handling.

Testing

The project includes tests. Ensure you have installed the test dependencies (pip install -e .[test] or uv pip install -e .[test]).

Run tests using pytest:

pytest tests

Note: Tests might require mocking Redis (fakeredis) and potentially the OpenRouter API calls depending on their implementation.

Troubleshooting

• Error: OpenRouter API key not available: Ensure OPENROUTER_API_KEY is set or ~/.api-openrouter exists with your key on a single line (no whitespace).
• Jobs stuck in "queued": Verify that the RQ worker (python -m mcp_waifu_queue.worker) is running in a separate terminal and connected to the same Redis instance specified in .env. Check the worker logs for errors.
• ConnectionRefusedError (Redis): Make sure your Redis server is running and accessible at the REDIS_URL specified in .env.
• MCP Server Connection Issues: Ensure the MCP server (uvicorn ...) is running and you are connecting to the correct host/port.

Contributing

1. Fork the repository.
2. Create a new branch for your feature or bug fix (git checkout -b feature/your-feature-name).
3. Make your changes and commit them (git commit -am 'Add some feature').
4. Push your branch to your forked repository (git push origin feature/your-feature-name).
5. Create a Pull Request on the original repository.

Please adhere to the project's coding standards and linting rules (ruff).

License

This project is licensed under the MIT-0 License - see the LICENSE file for details.