Whoishiring Server

An automated job discovery and tracking pipeline that scrapes multiple sources, applies a multi-stage filter cascade, and exposes a 24-tool MCP server so Claude Desktop can analyze job fit against a resume — all without requiring an API key.

What It Does

Aggregates job postings from two sources (Hacker News "Who is hiring?" threads via Algolia API, and YC's Work at a Startup board via Playwright headless browser), runs them through a shared multi-stage filter pipeline, maintains a bounded tracking system with backlog promotion, and provides a full MCP (Model Context Protocol) tool interface for an LLM to analyze, rank, and manage a job search pipeline end-to-end.

Architecture

  Algolia API ──→ hn_jobs.py ──┐
                               ├──→ filters.py ──→ mcp_server.py ──→ Claude Desktop
  Playwright  ──→ waas.py   ──┘         │              │
                                    Shared filter    24 MCP tools
                                    pipeline         8 JSON stores
                                                     Description cache

Multi-Source Ingestion

• HN threads: Algolia search API → parallelized comment fetch (20-worker ThreadPoolExecutor) → HTML comment parsing to extract company, location, remote status, emails, job board URLs
• WAAS: Playwright headless browser with authenticated session → infinite-scroll scraping → structured field extraction (title, salary, batch, company size, seniority)
• Both sources feed into the same filters.py pipeline — keyword scoring, negative filters, seniority estimation, job type classification, and location detection share a single implementation

Filter Pipeline (filters.py)

Five-stage filter cascade, each with configurable behavior:

1. Weighted keyword scoring — Three categories scored by relevance (AI tooling: 3, Systems: 2, General AI+SWE: 1). Scoring is per-category, not per-keyword — multiple hits in one category don't stack. All matching uses compiled \b word-boundary regex, case-insensitive.

2. Negative keyword filter — Detects senior/management titles (staff, principal, director, VP) and high experience thresholds (10+, 15+ years). Matched posts aren't silently dropped — they go to a "Filtered Out" section so nothing is lost.

3. Seniority estimation — Infers seniority from job title keywords and description experience-year requirements. Maps to a level scale (intern → junior → mid → senior → staff+). Configurable max level — jobs above the threshold are filtered. Unknown seniority is never filtered (benefit of the doubt).

4. Job type classification — Classifies roles as coding (engineer, developer, SRE, etc.) vs non-coding (PM, designer, sales, recruiter). Engineering management is classified as non-coding. Unknown types are kept.

5. Location filter — Regex matching against known US and non-US cities/countries. Non-US, non-remote jobs are filtered. No detected location = kept (benefit of the doubt).

Bounded Job Tracking System

Eight JSON files managed atomically (read-modify-write) by the MCP server:

scan_waas ──→ [all new jobs] ──→ backlog_jobs.json (overflow, ranked by score)
                                        │
                                        ▼ (top N promoted)
                                 tracked_jobs.json (max N active, default 20)
                                        │
                          ┌──────────────┼──────────────┐
                          ▼              ▼              ▼
                   applied_jobs    dismissed_jobs  longshot_jobs
                   (permanent)     (validated)     (validated)
                          │
                    ┌─────┴─────┐
                    ▼           ▼
             rejected_jobs  accepted_jobs
             (outcome)      (outcome)

• Tracked — Bounded active pipeline (max N, configurable). Only the highest-scoring jobs from backlog fill these slots. Every mark_applied, mark_dismissed, or mark_longshot call frees a slot and auto-promotes the top backlog entry, returning the promoted job in the response to avoid redundant get_tracked_jobs calls.
• Backlog — Unbounded overflow, sorted by keyword score. Jobs that pass all filters but don't make the top N cut. Promotion happens automatically when tracked slots open.
• Applied — Permanent record. Not validated against WAAS (job might be filled but the application record matters).
• Dismissed / Longshot — Validated periodically against WAAS — dead listings are pruned automatically.
• Rejected / Accepted — Outcome tracking for applied jobs. mark_rejected and mark_accepted move jobs from applied to their respective stores. Both can be reverted to tracked via mark_open.

Description Caching

Full job descriptions are cached to disk during scanning (job_descriptions.json). get_job_details reads from cache first, falls back to live HTTP + BeautifulSoup parsing. Cache is auto-pruned — descriptions are removed when jobs leave all active stores (tracked, backlog, applied, longshot, rejected, accepted). Applied/rejected/accepted jobs retain their descriptions permanently.

MCP Server (24 Tools)

Stdio-transport MCP server for Claude Desktop. Claude Desktop acts as the LLM ranker, eliminating the need for an API key. The server owns all state — Claude never writes to JSON files directly, only through tool calls.

Scanning: scan_jobs, scan_waas, scan_all, get_job_details

• scan_waas returns only run metadata (counts, timing, active filters) — not job data. Job data flows through the tracking system.
• scan_all combines HN + WAAS with cross-source dedup by company name (case-insensitive, whitespace-stripped). HN takes priority.
• get_job_details serves from disk cache, falls back to live fetch with structured extraction (JSON-LD → Open Graph → meta tags → title tag).

Tracking: get_tracked_jobs, get_applied_jobs, get_dismissed_jobs, get_longshot_jobs, get_rejected_jobs, get_accepted_jobs, update_job_analysis, mark_applied, mark_dismissed, mark_longshot, mark_rejected, mark_accepted, mark_open, swap_role, validate_tracked_jobs, reset_tracking

• swap_role replaces a tracked job with an alternate role URL from the same company (e.g., a better-fit position discovered through other_roles), clearing stale analysis.
• validate_tracked_jobs checks open/dismissed/longshot jobs against WAAS, removes dead listings, and backfills from backlog.
• mark_applied/mark_dismissed/mark_longshot free a tracked slot and return the newly promoted backlog job inline, enabling a dismiss-and-analyze loop without re-fetching the full tracked list.
• mark_rejected/mark_accepted move applied jobs to outcome tracking. Both can be reverted via mark_open.

Config: get_resume, get_preferences, get_config, update_config, get_latest_results

• get_resume extracts text from a configured PDF via PyMuPDF.
• update_config writes to config.yaml for runtime filter/preference changes.

Daily Workflow (Automated via MCP Prompt)

1. validate_tracked_jobs — prune dead listings from tracked/dismissed/longshot, backfill from backlog
2. scan_waas — discover new jobs, auto-track top N by score, overflow to backlog
3. get_tracked_jobs → for each unanalyzed job: get_job_details → update_job_analysis (or mark_dismissed/mark_longshot with inline backfill loop)
4. Render tracked/applied/dismissed/longshot/rejected/accepted into a React artifact

Deduplication

• Within-source: seen_posts.json (HN comment IDs), seen_waas.json (WAAS job URLs). Auto-pruned after 6 months.
• Cross-source: scan_all deduplicates by company name (case-insensitive, stripped). HN takes priority since it has richer context.
• First-run backfill: Empty seen files trigger a 3-month HN backfill scan.

Tech Stack

Component	Technology	Purpose
HN ingestion	requests + Algolia API	Parallelized comment fetching (20-worker ThreadPoolExecutor)
WAAS scraping	playwright	Headless browser with auth, infinite scroll handling
HTML parsing	beautifulsoup4	Job board scraping (Greenhouse, Lever, Ashby)
Resume parsing	pymupdf	PDF text extraction
Config	pyyaml	YAML config with runtime updates
LLM integration	anthropic (CLI) / MCP stdio (Desktop)	Resume-based job ranking
MCP server	mcp (FastMCP)	20-tool stdio server for Claude Desktop
Filter pipeline	re (compiled word-boundary regex)	Shared across both sources
State management	JSON files (atomic read-modify-write)	8 tracking stores + description cache

Setup

python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
cp config.yaml.example config.yaml  # edit with your preferences

Environment Variables (.env, gitignored)

Variable	Purpose
ANTHROPIC_API_KEY	Claude API (CLI ranking mode only)
WAAS_USERNAME / WAAS_PASSWORD	YC account for full WAAS access (~30 jobs without)
HN_JOBS_EMAIL_TO / FROM / PASSWORD	Email delivery (Gmail app password)
TRACKING_DIR	Directory for tracking JSON files (keeps paths out of git)

Claude Desktop MCP Config

{
  "mcpServers": {
    "hn-jobs": {
      "type": "stdio",
      "command": "wsl",
      "args": [
        "bash", "-c",
        "set -a; source /path/to/.env; set +a; /path/to/.venv/bin/python3 /path/to/mcp_server.py"
      ]
    }
  }
}

CLI Usage

python3 hn_jobs.py --dry-run --no-rank   # HTML preview, no ranking
python3 hn_jobs.py --no-email             # Terminal output
python3 hn_jobs.py --dry-run --resume resume.pdf  # Rank against resume
python3 hn_jobs.py                        # Full scan + email delivery