Metalsmarket Bot

A self-hosted toolkit that collects metal & commodity price data from metalsmarket.net and serves it to any MCP-capable LLM client (Claude Desktop, Cursor, or an HTTP MCP host) through a small Model Context Protocol server.

[!WARNING] Responsible use. This tool automates login and data collection against a third-party commercial site. Use only your own account, respect metalsmarket.net's Terms of Service and rate limits, and do not redistribute the scraped price data. It is provided for educational and personal use, with no warranty. You are responsible for how you use it.

---

What it does

There are two independent data paths plus an MCP server that reads their output (the MCP server never talks to the site directly):

Component	File	Produces	Runs
Live collector	collector.py → metalsmarket_bot.py	data/latest_snapshot.json, data/history.csv	Continuous loop, random 40–60 min interval (+ on-demand)
Historical factory	metals_master_factory.py	Metals_Historical_Full_2026.xlsx (one sheet per instrument)	One-shot backfill
MCP server	mcp_server.py	3 tools over stdio or Streamable HTTP	Reads the files above

MCP tools

• metalsmarket_canli_fiyat — live/last price for a metal or instrument (Turkish/English aliases: altın→gold, çinko→zinc, bakır→copper, …).
• metalsmarket_gecmis_fiyat — historical price for a single date or a date range/trend, from the backfill workbook.
• metalsmarket_veri_durumu — freshness/health of the collected data.

Key features

• Single POST per round pulls the entire dashboard (~580 cells).
• Randomized polling (no fixed period) to avoid a predictable request cadence.
• On-demand refresh — a live query older than ~2 minutes triggers a single, coalesced, throttled refresh through the collector's session.
• Resilient login — on a session kick it re-logs in automatically; backs off only on HTTP 429 (5/15/30 min escalating) to stay rate-limit friendly.
• Two MCP transports — stdio (Claude Desktop) and Streamable HTTP (URL-based hosts). Optional JWT auth + localhost-only binding for access control.

Requirements

• Python 3.11+
• A valid metalsmarket.net account
• pip install -r requirements.txt

Setup

# 1. install dependencies
python -m pip install -r requirements.txt

# 2. configure credentials
cp .env.example .env        # then edit .env with your username/password

# 3. (first run only) build the cell-name metadata + run the collector
#    Windows:
start.bat                   # installs deps, builds metadata, starts the loop

index.html (the site's Historical symbol dropdown, included as the instrument catalogue) is required by the collector and the historical factory.

Usage

# Live collector — continuous loop (random 40-60 min, refreshes on demand)
python collector.py
python collector.py --once          # single round then exit

# Historical backfill — one-shot scrape into the .xlsx workbook
python metals_master_factory.py
python metals_master_factory.py --debug-one <symbol_id>   # verify the parser first

# MCP server
python mcp_server.py                 # stdio (Claude Desktop / Cursor)
python mcp_server.py --http --port 3011   # Streamable HTTP at /mcp

Connecting an HTTP MCP host

The HTTP server binds to 127.0.0.1 by default, so only the local machine (and Docker Desktop containers via host.docker.internal) can reach it — the LAN and the public internet cannot. Point your MCP host at:

http://host.docker.internal:3011/mcp     # from a Docker container
http://127.0.0.1:3011/mcp                 # from the host

For token-based access control, set MCP_JWT_SECRET in .env; the endpoint then requires Authorization: Bearer <HS256 JWT> and rejects everything else with 401.

Project layout

collector.py              live collector loop (+ on-demand refresh)
metalsmarket_bot.py       login + single-POST dashboard fetch + resilience
csv_writer.py             snapshot.json / history.csv writers
mcp_server.py             MCP server (stdio + Streamable HTTP, 3 tools)
metals_master_factory.py  one-shot historical backfill -> .xlsx
cell_metadata_factory.py  builds cell-id -> name metadata (setup, HTTP)
cell_metadata_pw.py       Playwright supplement for JS-only widgets (setup)
index.html                instrument catalogue (site symbol dropdown)
start.bat / factory.bat / mcp_http.bat   Windows launchers

Scraped data (data/, *.xlsx, scraped_ids.txt, debug_pages/) and .env are git-ignored and never published.