Deriva Server

Model Context Protocol (MCP) server that exposes Deriva catalog operations and DerivaML ML workflow tools for LLM applications.

Overview

This MCP server provides an interface to Deriva catalogs and DerivaML, enabling AI assistants like Claude to:

• Connect to and manage Deriva catalogs
• Create and manage datasets with versioning
• Work with controlled vocabularies
• Define and execute ML workflows
• Create and manage features for ML experiments

For full ML workflow management, this server is designed to work alongside the GitHub MCP Server to enable:

• Storing and versioning hydra-zen configurations in GitHub repositories
• Managing workflow code and model implementations
• Collaborative development of ML experiments

Prerequisites

Deriva Authentication

DerivaML uses Globus for authentication. Before using the MCP server, you must authenticate with your Deriva server:

# Install deriva-ml if not already installed
pip install deriva-ml

# Authenticate with your Deriva server
python -c "from deriva_ml import DerivaML; DerivaML.globus_login('your-server.org')"

This opens a browser window for Globus authentication. Credentials are cached locally and persist across sessions.

Alternatively, use the Deriva Auth Agent for browser-based authentication:

1. Install the Deriva Auth Agent from deriva-py
2. Run deriva-globus-auth-utils login --host your-server.org

GitHub Authentication (for configuration management)

Create a GitHub Personal Access Token (PAT) for the GitHub MCP Server:

1. Go to GitHub Settings > Personal Access Tokens
2. Create a fine-grained token with these permissions:
   • Repository access: Select repositories containing your ML configurations
   • Permissions:
     • Contents: Read and write (for pushing configs)
     • Pull requests: Read and write (optional, for PR workflows)
     • Issues: Read (optional, for tracking)
3. Copy the token securely - you'll need it for configuration

Installation

Using Docker (Recommended)

Docker provides the simplest setup with no Python environment management. The image is automatically built and published to GitHub Container Registry on every commit to main.

# Pull the latest image from GitHub Container Registry
docker pull ghcr.io/informatics-isi-edu/deriva-mcp:latest

# Run the server (for testing)
docker run --rm -it ghcr.io/informatics-isi-edu/deriva-mcp:latest --help

To build locally instead:

git clone https://github.com/informatics-isi-edu/deriva-mcp.git
cd deriva-mcp
./scripts/docker-build.sh

Using uv

uv pip install deriva-mcp

Using pip

pip install deriva-mcp

From source

git clone https://github.com/informatics-isi-edu/deriva-mcp.git
cd deriva-mcp
uv sync

Claude Code Plugin (Skills)

The Deriva skills plugin for Claude Code provides 30+ skills that guide Claude through common Deriva and DerivaML workflows. The skills are maintained in a separate repository: deriva-skills.

Installing the Plugin

# Add the marketplace (one-time)
/plugin marketplace add informatics-isi-edu/deriva-skills

# Install the plugin
/plugin install deriva

Updating the Plugin

To update to the latest version:

/plugin install deriva

Or check your entire DerivaML ecosystem:

/deriva:check-versions

This checks three components — the deriva-ml Python package, the deriva-skills plugin, and the deriva-mcp MCP server — against upstream releases and offers to update outdated ones.

See the deriva-skills README for the full list of available skills.

Development: Testing Skills Locally

During development, load the plugin from a local path without installing:

claude --plugin-dir /path/to/deriva-skills

Configuration

Claude Desktop - Full Setup with GitHub Integration

For the complete ML workflow experience, configure both DerivaML and GitHub MCP servers together.

Configuration file locations:

• macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
• Windows: %APPDATA%\Claude\claude_desktop_config.json
• Linux: ~/.config/Claude/claude_desktop_config.json

Option 1: Both Servers with Docker (Recommended)

Uses Docker for both MCP servers - most consistent setup:

{
  "mcpServers": {
    "deriva": {
      "type": "stdio",
      "command": "/bin/sh",
      "args": [
        "-c",
        "docker run -i --rm --add-host localhost:host-gateway -e HOME=$HOME -v $HOME/.deriva:$HOME/.deriva:ro -v $HOME/.bdbag:$HOME/.bdbag -v $HOME/.deriva-ml:$HOME/.deriva-ml ghcr.io/informatics-isi-edu/deriva-mcp:latest"
      ],
      "env": {}
    },
    "github": {
      "type": "stdio",
      "command": "docker",
      "args": [
        "run", "-i", "--rm",
        "-e", "GITHUB_PERSONAL_ACCESS_TOKEN",
        "ghcr.io/github/github-mcp-server"
      ],
      "env": {
        "GITHUB_PERSONAL_ACCESS_TOKEN": "ghp_your_token_here"
      }
    }
  }
}

Docker arguments explained:

• --add-host localhost:host-gateway - Allows connecting to a Deriva server running on localhost
• -e HOME=$HOME - Passes your home directory path into the container so mounted paths are found correctly

For localhost with self-signed certificates, the image defaults to using ~/.deriva/allCAbundle-with-local.pem as the CA bundle. See Troubleshooting for how to create this file.

Volume mounts explained:

• $HOME/.deriva:$HOME/.deriva:ro - Mounts your Deriva credentials (read-only)
• $HOME/.bdbag:$HOME/.bdbag - Mounts bdbag keychain for dataset download authentication (writable)
• $HOME/.deriva-ml:$HOME/.deriva-ml - Working directory for execution outputs (writable)

Note: Create the workspace directory before first use:

mkdir -p ~/.deriva-ml

If the directory doesn't exist, Docker creates it as root, causing permission issues.

Option 2: Direct Install with GitHub Remote

Uses pip-installed DerivaML MCP with GitHub's hosted server:

{
  "mcpServers": {
    "deriva": {
      "type": "stdio",
      "command": "deriva-mcp",
      "env": {}
    },
    "github": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "@anthropic-ai/github-mcp-server"],
      "env": {
        "GITHUB_PERSONAL_ACCESS_TOKEN": "ghp_your_token_here"
      }
    }
  }
}

Option 3: From Source (Development)

For development or customization:

{
  "mcpServers": {
    "deriva": {
      "type": "stdio",
      "command": "uv",
      "args": [
        "--directory",
        "/path/to/deriva-mcp",
        "run",
        "deriva-mcp"
      ],
      "env": {}
    },
    "github": {
      "type": "stdio",
      "command": "docker",
      "args": [
        "run", "-i", "--rm",
        "-e", "GITHUB_PERSONAL_ACCESS_TOKEN",
        "ghcr.io/github/github-mcp-server"
      ],
      "env": {
        "GITHUB_PERSONAL_ACCESS_TOKEN": "ghp_your_token_here"
      }
    }
  }
}

Option 4: DerivaML Only (No GitHub)

If you don't need GitHub integration:

{
  "mcpServers": {
    "deriva": {
      "type": "stdio",
      "command": "/bin/sh",
      "args": [
        "-c",
        "docker run -i --rm --add-host localhost:host-gateway -e HOME=$HOME -v $HOME/.deriva:$HOME/.deriva:ro -v $HOME/.bdbag:$HOME/.bdbag -v $HOME/.deriva-ml:$HOME/.deriva-ml ghcr.io/informatics-isi-edu/deriva-mcp:latest"
      ],
      "env": {}
    }
  }
}

Or with direct install:

{
  "mcpServers": {
    "deriva": {
      "type": "stdio",
      "command": "deriva-mcp",
      "env": {}
    }
  }
}

Claude Code

Add to ~/.mcp.json (global) or your project's .mcp.json file:

With Docker:

{
  "mcpServers": {
    "deriva": {
      "type": "stdio",
      "command": "/bin/sh",
      "args": [
        "-c",
        "docker run -i --rm --add-host localhost:host-gateway -e HOME=$HOME -v $HOME/.deriva:$HOME/.deriva:ro -v $HOME/.bdbag:$HOME/.bdbag -v $HOME/.deriva-ml:$HOME/.deriva-ml ghcr.io/informatics-isi-edu/deriva-mcp:latest"
      ],
      "env": {}
    },
    "github": {
      "type": "stdio",
      "command": "docker",
      "args": [
        "run", "-i", "--rm",
        "-e", "GITHUB_PERSONAL_ACCESS_TOKEN",
        "ghcr.io/github/github-mcp-server"
      ],
      "env": {
        "GITHUB_PERSONAL_ACCESS_TOKEN": "${GITHUB_PERSONAL_ACCESS_TOKEN}"
      }
    }
  }
}

With direct install:

{
  "mcpServers": {
    "deriva": {
      "type": "stdio",
      "command": "deriva-mcp",
      "env": {}
    },
    "github": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "@anthropic-ai/github-mcp-server"],
      "env": {
        "GITHUB_PERSONAL_ACCESS_TOKEN": "ghp_your_token_here"
      }
    }
  }
}

Then enable in .claude/settings.local.json:

{
  "enableAllProjectMcpServers": true,
  "enabledMcpjsonServers": ["deriva", "github"]
}

HTTP Transport Mode (for Long-Running Operations)

For operations that take more than a few minutes (like catalog cloning), HTTP transport provides a persistent connection that survives client disconnects. The server runs as a background service and maintains task state across reconnections.

Starting the HTTP Server

Using Docker Compose (Recommended):

# Requires the deriva-localhost Docker network for localhost catalogs
docker-compose -f docker-compose.mcp.yaml up -d

# View logs
docker-compose -f docker-compose.mcp.yaml logs -f

# Stop the server
docker-compose -f docker-compose.mcp.yaml down

Using Docker directly:

docker run -d --name deriva-mcp \
  -p 8000:8000 \
  --network deriva-localhost_internal_network \
  -e HOME=$HOME \
  -e DERIVA_MCP_LOCALHOST_ALIAS=deriva-webserver \
  -v $HOME/.deriva:$HOME/.deriva:ro \
  -v $HOME/.bdbag:$HOME/.bdbag \
  -v $HOME/.deriva-ml:$HOME/.deriva-ml \
  ghcr.io/informatics-isi-edu/deriva-mcp:latest \
  deriva-mcp --transport streamable-http --host 0.0.0.0 --port 8000

The DERIVA_MCP_LOCALHOST_ALIAS variable tells the entrypoint to resolve the Docker DNS name deriva-webserver to an IP and add it to /etc/hosts as localhost. This avoids hardcoding container IPs that change when the network is recreated.

Running locally (development):

# From source
uv run deriva-mcp --transport streamable-http --host 0.0.0.0 --port 8000

# If installed via pip
deriva-mcp --transport streamable-http --host 0.0.0.0 --port 8000

Client Configuration for HTTP

Claude Code (~/.mcp.json):

{
  "mcpServers": {
    "deriva": {
      "type": "http",
      "url": "http://localhost:8000/mcp"
    }
  }
}

Claude Desktop:

{
  "mcpServers": {
    "deriva": {
      "type": "http",
      "url": "http://localhost:8000/mcp"
    }
  }
}

Benefits of HTTP Transport

• No idle timeouts: Long-running operations complete without connection drops
• Persistent server: Server survives client disconnects and restarts
• Task state preservation: Background tasks continue even if you close Claude
• Multiple clients: Multiple Claude sessions can share the same server
• Health checks: Docker can automatically restart unhealthy servers

Verifying the HTTP Server

# Check server health (uses /health endpoint, not /mcp which creates sessions)
curl -s http://localhost:8000/health

# View server logs
docker-compose -f docker-compose.mcp.yaml logs -f

VS Code with Continue or Cline

Add to your MCP configuration (typically .vscode/mcp.json):

{
  "mcp": {
    "servers": {
      "deriva": {
        "type": "stdio",
        "command": "/bin/sh",
        "args": [
          "-c",
          "docker run -i --rm --add-host localhost:host-gateway -e HOME=$HOME -v $HOME/.deriva:$HOME/.deriva:ro -v $HOME/.bdbag:$HOME/.bdbag -v $HOME/.deriva-ml:$HOME/.deriva-ml ghcr.io/informatics-isi-edu/deriva-mcp:latest"
        ],
        "env": {}
      },
      "github": {
        "type": "stdio",
        "command": "docker",
        "args": [
          "run", "-i", "--rm",
          "-e", "GITHUB_PERSONAL_ACCESS_TOKEN",
          "ghcr.io/github/github-mcp-server"
        ],
        "env": {
          "GITHUB_PERSONAL_ACCESS_TOKEN": "ghp_your_token_here"
        }
      }
    }
  }
}

Environment Variables

For security, store tokens in environment variables instead of config files:

# Add to ~/.bashrc, ~/.zshrc, or equivalent
export GITHUB_PERSONAL_ACCESS_TOKEN="ghp_your_token_here"

Then reference in config:

{
  "mcpServers": {
    "github": {
      "type": "stdio",
      "command": "docker",
      "args": ["run", "-i", "--rm", "-e", "GITHUB_PERSONAL_ACCESS_TOKEN", "ghcr.io/github/github-mcp-server"],
      "env": {
        "GITHUB_PERSONAL_ACCESS_TOKEN": "${GITHUB_PERSONAL_ACCESS_TOKEN}"
      }
    }
  }
}

Verifying Your Setup

After configuration, verify both servers are working:

User: What MCP servers are available?

Claude: I have access to two MCP servers:
1. deriva - For managing ML workflows in Deriva catalogs
2. github - For managing GitHub repositories and configurations

User: Connect to the deriva catalog at example.org with ID 42

Claude: [Uses connect_catalog tool]
Connected to example.org, catalog 42. The domain schema is 'my_project'.

User: List the hydra-zen configs in the my-ml-project repo

Claude: [Uses GitHub get_file_contents tool]
Found configuration files in configs/:
- deriva.py - DerivaML connection settings
- datasets.py - Dataset specifications
- model.py - Model hyperparameters

Available Tools

Catalog Management

Tool	Description
connect_catalog	Connect to a DerivaML catalog
disconnect_catalog	Disconnect from the active catalog
list_connections	List all active connections
set_active_catalog	Set which connection is active
get_catalog_info	Get information about the active catalog
list_users	List users with catalog access
get_chaise_url	Get web interface URL for a table
resolve_rid	Find which table a RID belongs to
list_catalog_registry	List all catalogs and aliases on a server
create_catalog	Create a new DerivaML catalog (with optional alias)
delete_catalog	Permanently delete a catalog
clone_catalog	Clone a catalog to create a copy

Catalog Alias Management

Tool	Description
create_catalog_alias	Create an alias for a catalog
get_catalog_alias	Get alias metadata (target, owner)
update_catalog_alias	Update alias target or owner
delete_catalog_alias	Delete an alias (catalog not affected)

Dataset Management

Tool	Description
find_datasets	Find all datasets in the catalog
lookup_dataset	Look up detailed information about a dataset
create_dataset	Create a new dataset
list_dataset_members	List members of a dataset
add_dataset_members	Add members to a dataset
get_dataset_version_history	Get version history
increment_dataset_version	Update dataset version
delete_dataset	Delete a dataset
list_dataset_element_types	List valid element types
add_dataset_element_type	Enable a table as element type

Vocabulary Management

Tool	Description
list_vocabularies	List all vocabulary tables
list_vocabulary_terms	List terms in a vocabulary
lookup_term	Find a term by name or synonym
add_term	Add a term to a vocabulary
create_vocabulary	Create a new vocabulary table

Workflow Management

Tool	Description
find_workflows	Find all workflows
lookup_workflow	Find a workflow by URL/checksum
create_workflow	Create and register a workflow
list_workflow_types	List available workflow types
add_workflow_type	Add a new workflow type

Feature Management

Tool	Description
find_features	Find features for a table
lookup_feature	Get feature details
list_feature_values	Get all values for a feature
create_feature	Create a feature definition
delete_feature	Delete a feature
list_feature_names	List all feature names

Schema Management

Tool	Description
create_table	Create a new table in the domain schema
create_asset_table	Create an asset table for file management
list_assets	List all assets in an asset table
list_tables	List all tables in the domain schema
get_table_schema	Get column and key definitions for a table
list_asset_types	List available asset type terms
add_asset_type	Add a new asset type to the vocabulary

Execution Management

Tool	Description
create_execution	Create a new execution for ML workflows
start_execution	Start the active execution
stop_execution	Stop and complete the active execution
update_execution_status	Update execution status and message
get_execution_info	Get details about the active execution
restore_execution	Restore a previous execution by RID
asset_file_path	Register a file for upload as an execution output
commit_output_assets	Commit all registered outputs to the catalog
list_executions	List recent executions
create_execution_dataset	Create a dataset within an execution
download_execution_dataset	Download a dataset for processing
get_execution_working_dir	Get the working directory path

Execution Workflow

The typical execution workflow using the context manager:

with execution.execute() as exe:
    # Do your work here
    exe.asset_file_path(asset_name="Image", file_name="output.png")
    # ... more processing ...

# After context exits, commit output assets
execution.commit_output_assets()

Using MCP tools, the equivalent workflow is:

1. create_execution() - Create the execution record with workflow info
2. start_execution() - Mark execution as running, begin timing
3. asset_file_path() - Register output files (repeat as needed)
4. stop_execution() - Mark execution as complete
5. commit_output_assets() - Required: Commit all registered files to catalog

Important: You must call commit_output_assets() after completing your work to commit any registered assets to the catalog. This is not automatic.

Available Resources

MCP resources provide read-only access to catalog information and configuration templates.

Static Resources - Configuration Templates

These resources provide code templates for configuring DerivaML with hydra-zen:

Resource URI	Description
deriva://config/deriva-ml-template	Hydra-zen configuration template for DerivaML connection
deriva://config/dataset-spec-template	Configuration template for dataset specifications
deriva://config/execution-template	Configuration template for ML executions
deriva://config/model-template	Configuration template for ML models with zen_partial

Dynamic Resources - Catalog Information

These resources return current catalog state (requires active connection):

Resource URI	Description
deriva://catalog/schema	Current catalog schema structure in JSON
deriva://catalog/vocabularies	All vocabulary tables and their terms
deriva://catalog/datasets	All datasets in the current catalog
deriva://catalog/workflows	All registered workflows
deriva://catalog/features	All feature names defined in the catalog

Template Resources - Parameterized

These resources accept parameters to return specific information:

Resource URI	Description
deriva://dataset/{dataset_rid}	Detailed information about a specific dataset
deriva://table/{table_name}/features	Features defined for a specific table
deriva://vocabulary/{vocab_name}	Terms in a specific vocabulary table

Documentation Resources

Documentation is fetched dynamically from GitHub repositories with 1-hour caching:

Resource URI	Description
deriva://docs/overview	DerivaML overview and architecture
deriva://docs/datasets	Guide to creating and managing datasets
deriva://docs/features	Guide to defining and using features
deriva://docs/execution-configuration	Guide to configuring ML executions
deriva://docs/hydra-zen	Guide to hydra-zen configuration
deriva://docs/file-assets	Guide to managing file assets
deriva://docs/notebooks	Guide to Jupyter notebook integration
deriva://docs/identifiers	Guide to RIDs, MINIDs, and identifiers
deriva://docs/install	Installation instructions
deriva://docs/ermrest/*	ERMrest API documentation
deriva://docs/chaise/*	Chaise UI documentation
deriva://docs/deriva-py/*	Deriva Python SDK documentation

Using Resources

Resources are accessed differently than tools - they provide static or semi-static data that can be read without side effects:

User: Show me the DerivaML configuration template

Claude: [Reads deriva://config/deriva-ml-template resource]
Here's a hydra-zen configuration template for DerivaML...

User: What datasets are in the catalog?

Claude: [Reads deriva://catalog/datasets resource]
Found the following datasets in your catalog...

Usage Examples

Discovering and Connecting to Catalogs

User: What catalogs are available on example.org?

Claude: [Uses list_catalog_registry tool]
Found 3 catalogs on example.org:
- ID: 21, Name: "ML Project", Persistent: true
- ID: 45, Name: "Test Environment", Persistent: true
- ID: 50, Name: "Clone of ML Project", Persistent: true

And 2 aliases:
- "ml-project" -> catalog 21
- "test-env" -> catalog 45

User: Connect to ml-project

Claude: [Uses connect_catalog tool with catalog_id="ml-project"]
Connected to example.org using alias "ml-project" (catalog 21).
The domain schema is 'my_domain'.

Cloning and Managing Catalogs

User: Clone the production catalog to create a test environment

Claude: [Uses clone_catalog tool]
Cloned catalog 21 to new catalog 52 with all data and policies.

User: Create an alias "dev-test" for the new catalog

Claude: [Uses create_catalog_alias tool]
Created alias "dev-test" pointing to catalog 52.
You can now connect using: connect_catalog("example.org", "dev-test")

Basic Catalog Operations

User: Connect to the deriva catalog at example.org with ID 123

Claude: I'll connect to that catalog for you.
[Uses connect_catalog tool]
Connected to example.org, catalog 123. The domain schema is 'my_project'.

User: What datasets are available?

Claude: Let me check what datasets exist.
[Uses find_datasets tool]
Found 5 datasets:
1. Training Images (v1.2.0) - 1500 images for model training
2. Validation Set (v1.0.0) - 300 images for validation
...

Configuration Workflow with GitHub

User: Create a new hydra-zen config for training on the cell-images dataset

Claude: I'll create a configuration file for you.
[Reads deriva://config/model-template resource for template]
[Uses GitHub create_or_update_file to save configs/cell_training.py]

Created configs/cell_training.py with:
- DerivaML connection to example.org
- Dataset spec for cell-images v2.0.0
- Model config with learning_rate=1e-3, epochs=50

User: Commit and push that config

Claude: [Uses GitHub create_or_update_file with commit message]
Committed and pushed to main branch.

Hydra-zen Configuration

DerivaML integrates with hydra-zen for configuration management, enabling reproducible ML workflows with structured configuration.

Basic Configuration

from hydra_zen import builds, instantiate
from deriva_ml import DerivaML
from deriva_ml.core.config import DerivaMLConfig

# Create a structured config using hydra-zen
DerivaMLConf = builds(DerivaMLConfig, populate_full_signature=True)

# Configure for your environment
conf = DerivaMLConf(
    hostname='deriva.example.org',
    catalog_id='42',
    domain_schema='my_domain',
)

# Instantiate to get a DerivaMLConfig object, then create DerivaML
config = instantiate(conf)
ml = DerivaML.instantiate(config)

Working Directory Configuration

DerivaML automatically configures Hydra's output directory based on your working_dir setting:

conf = DerivaMLConf(
    hostname='deriva.example.org',
    working_dir='/shared/ml_workspace',  # Custom working directory
)

Hydra outputs will be organized under: {working_dir}/{username}/deriva-ml/hydra/{timestamp}/

Configuration Composition

Create environment-specific configurations using hydra-zen's store:

from hydra_zen import store

# Development configuration
store(DerivaMLConf(
    hostname='dev.example.org',
    catalog_id='1',
), name='dev')

# Production configuration
store(DerivaMLConf(
    hostname='prod.example.org',
    catalog_id='100',
), name='prod')

Dataset Specification Configuration

Use DatasetSpecConfig for cleaner dataset specifications:

from deriva_ml.dataset import DatasetSpecConfig

# Create dataset specs (hydra-zen compatible)
training_data = DatasetSpecConfig(
    rid="1ABC",
    version="1.0.0",
    materialize=True,       # Download asset files
    description="Training images"
)

metadata_only = DatasetSpecConfig(
    rid="2DEF",
    version="2.0.0",
    materialize=False,      # Only download table data
)

# Use in hydra-zen store
from hydra_zen import store
datasets_store = store(group="datasets")
datasets_store([training_data], name="training")
datasets_store([metadata_only], name="metadata_only")

Asset Configuration

Use AssetRIDConfig for input assets (model weights, config files):

from deriva_ml.execution import AssetRIDConfig

# Define input assets
model_weights = AssetRIDConfig(rid="WXYZ", description="Pretrained model")
config_file = AssetRIDConfig(rid="ABCD", description="Hyperparameters")

# Store asset collections
assets_store = store(group="assets")
assets_store([model_weights, config_file], name="default_assets")

Execution Configuration

Configure ML executions with ExecutionConfiguration:

from hydra_zen import builds, instantiate
from deriva_ml.execution import ExecutionConfiguration
from deriva_ml.dataset import DatasetSpecConfig

# Build execution config
ExecConf = builds(ExecutionConfiguration, populate_full_signature=True)

# Configure execution with datasets and assets
conf = ExecConf(
    description="Training run",
    datasets=[
        DatasetSpecConfig(rid="1ABC", version="1.0.0", materialize=True),
    ],
    assets=["WXYZ", "ABCD"],  # Asset RIDs
)

exec_config = instantiate(conf)

Configuration Summary

Class	Module	Purpose
DerivaMLConfig	deriva_ml.core.config	Main DerivaML connection config
DatasetSpecConfig	deriva_ml.dataset	Dataset specification for executions
AssetRIDConfig	deriva_ml.execution	Input asset specification
ExecutionConfiguration	deriva_ml.execution	Full execution configuration
Workflow	deriva_ml.execution	Workflow definition

See the DerivaML Hydra-zen Guide for complete documentation.

Troubleshooting

Docker with Localhost Deriva Server

When running the MCP server in Docker and connecting to a Deriva server on your local machine, you need additional configuration depending on how Deriva is running.

Option A: Deriva Running Directly on Host (not in Docker)

If your Deriva server is running directly on the host machine (not in Docker), use host-gateway:

{
  "mcpServers": {
    "deriva": {
      "type": "stdio",
      "command": "/bin/sh",
      "args": [
        "-c",
        "docker run -i --rm --add-host localhost:host-gateway -e HOME=$HOME -v $HOME/.deriva:$HOME/.deriva:ro -v $HOME/.bdbag:$HOME/.bdbag -v $HOME/.deriva-ml:$HOME/.deriva-ml ghcr.io/informatics-isi-edu/deriva-mcp:latest"
      ],
      "env": {}
    }
  }
}

Option B: Deriva Running in Docker (deriva-localhost)

If your Deriva server is running in Docker (e.g., using deriva-localhost), the MCP container must join the same Docker network and map localhost to the webserver container.

First, find the webserver IP:

docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' deriva-webserver

Then use it in the --add-host argument (replace <WEBSERVER_IP> with the actual IP):

{
  "mcpServers": {
    "deriva": {
      "type": "stdio",
      "command": "/bin/sh",
      "args": [
        "-c",
        "docker run -i --rm --network deriva-localhost_internal_network --add-host localhost:<WEBSERVER_IP> -e HOME=$HOME -v $HOME/.deriva:$HOME/.deriva:ro -v $HOME/.bdbag:$HOME/.bdbag -v $HOME/.deriva-ml:$HOME/.deriva-ml ghcr.io/informatics-isi-edu/deriva-mcp:latest"
      ],
      "env": {}
    }
  }
}

Why this is needed: The MCP container needs to download dataset assets from the Deriva server. When Deriva runs in Docker, URLs in the dataset bags reference localhost, which must resolve to the Deriva webserver container. The entrypoint script in the MCP image automatically adjusts /etc/hosts so that the --add-host mapping takes effect.

SSL Certificate Configuration

If your localhost Deriva server uses a self-signed certificate (common for development), the container won't trust it by default. The image automatically sets REQUESTS_CA_BUNDLE to $HOME/.deriva/allCAbundle-with-local.pem, so you just need to create this file:

Creating the CA bundle with your local certificate (macOS):

# Export the local CA certificate from System Keychain
security find-certificate -a -c "DERIVA Dev Local CA" -p /Library/Keychains/System.keychain > /tmp/deriva-local-ca.pem

# Combine with existing CA bundle (if you have one)
cat ~/.deriva/allCAbundle.pem /tmp/deriva-local-ca.pem > ~/.deriva/allCAbundle-with-local.pem

# Or just use the local CA alone
cp /tmp/deriva-local-ca.pem ~/.deriva/allCAbundle-with-local.pem

To use a different CA bundle path, override with -e REQUESTS_CA_BUNDLE=/path/to/bundle.pem.

Deriva Authentication Issues

Error: "No credentials found"

# Re-authenticate with Deriva
python -c "from deriva_ml import DerivaML; DerivaML.globus_login('your-server.org')"

Error: "Token expired"

# Force re-authentication
python -c "from deriva_ml import DerivaML; DerivaML.globus_login('your-server.org', force=True)"

GitHub MCP Issues

Error: "Bad credentials"

• Verify your PAT hasn't expired
• Check the token has required permissions (Contents: Read/Write)
• Ensure the token is correctly set in your config

Docker not found

• Install Docker Desktop or use the npx method instead
• On Linux, ensure your user is in the docker group

MCP Server Connection Issues

Server not responding

1. Check the server is installed: which deriva-mcp
2. Test manually: deriva-mcp (should start without errors)
3. Check Claude Desktop logs for errors

Multiple server conflicts

• Ensure each server has a unique name in the config
• Restart Claude Desktop after config changes

Long-Running Operations (Catalog Cloning)

Catalog cloning operations can take several minutes for large catalogs. The MCP server provides async tools (clone_catalog_async, get_task_status, list_tasks) to handle this.

Recommended: Use HTTP Transport for Long Operations

The HTTP transport mode solves connection timeout issues by running the server as a persistent service:

# Start the server with HTTP transport
docker-compose -f docker-compose.mcp.yaml up -d

# Or run directly
deriva-mcp --transport streamable-http --host 0.0.0.0 --port 8000

Then configure your MCP client to use HTTP:

{
  "mcpServers": {
    "deriva": {
      "type": "http",
      "url": "http://localhost:8000/mcp"
    }
  }
}

See HTTP Transport Mode for complete setup instructions.

Alternative: STDIO with Async Tools

If using STDIO transport, the async workflow helps manage timeouts:

1. Use clone_catalog_async() to start the clone - returns a task_id immediately
2. Periodically check get_task_status(task_id) for progress
3. When status is "completed", the result contains the new catalog info

STDIO Connection timeout issues: If connections drop during clone operations with STDIO transport:

1. Check task status after reconnection - If the MCP connection drops, simply reconnect and call list_tasks() to find your running tasks. Note: Task state is stored in memory, so if the server process restarts (not just reconnects), running tasks will be lost.

2. Consider switching to HTTP transport - HTTP connections don't have the idle timeout issues that STDIO connections have with some MCP clients.

Development

Running Tests

uv run pytest

Code Quality

uv run ruff check src/
uv run ruff format src/

Requirements

• Python 3.12+
• MCP SDK 1.2.0+
• DerivaML 0.1.0+
• Docker (optional, for GitHub MCP local server)

License

Apache 2.0

Related Projects

• DerivaML - Core library for ML workflows on Deriva
• Deriva - Python SDK for Deriva scientific data management
• GitHub MCP Server - Official GitHub MCP server
• MCP Python SDK - Official Python SDK for Model Context Protocol