Qa Ai Agents

An AI-powered QA automation pipeline built with TypeScript, Claude API, and the Model Context Protocol (MCP).

Architecture

Four specialised agents run in sequence, each grounded in feature requirements, business rules, and API contracts:

1. Agent 1 — Functional Tester: Generates test scenarios across happy path, negative, security, edge cases, UI states, and business rules
2. Agent 2 — Test Architect: Assigns each scenario to the correct test pyramid layer (unit / API / component / E2E), sets automation priority (P1/P2/P3), and flags manual-only scenarios
3. Agent 3 — Test Engineer: Writes production-grade Playwright tests for unit, API, and component layers using Page Object Model (POM). E2E tests are excluded — these are handled at SIT/UAT level
4. Agent 4 — Code Reviewer: Audits generated tests for selector quality, assertion completeness, and Playwright best practices

MCP Server

Exposes the pipeline as 4 MCP tools callable from Claude Desktop or Claude Code:

• run_qa_pipeline — full 4-agent pipeline
• generate_test_scenarios — Agent 1 only
• set_feature_input — write to input/feature.md
• get_pipeline_report — read last pipeline report

Tech Stack

• TypeScript + Node.js
• Anthropic Claude API (Sonnet 4.6 + Haiku 4.5)
• Ollama (local LLM support — toggle via USE_OLLAMA)
• Playwright (generated test output)
• Model Context Protocol (MCP) SDK

---

Setup

git clone https://github.com/abchahal/qa-ai-agents.git
cd qa-ai-agents
npm install
cp .env.example .env
# Add your ANTHROPIC_API_KEY to .env

---

Running the pipeline

# Via terminal
npm run pipeline

# Via Claude Desktop / Claude Code
# Type: "Run the full QA pipeline using input/feature.md"

---

Input files

File	Purpose
input/feature.md	Feature requirements, business rules, UI selectors
input/api-contract.json	API schema (optional)
input/source.js	Source code context (optional)

---

Output

output/
├── pages/             ← Page Object classes (one per page, shared across scenarios)
├── tests/
│   ├── unit/          ← Unit test spec files
│   ├── api/           ← API test spec files
│   └── component/     ← Component test spec files
└── pipeline_report.md

---

Model Strategy

Agent	Model	Reason
Agent 1	Haiku 4.5	Structured JSON output
Agent 2	Haiku 4.5	Classification task
Agent 3	Sonnet 4.6	Code generation — requires higher quality output
Agent 4	Haiku 4.5	Analysis and scoring

---

MCP Setup via CLI

Step 1 — Create the batch file

Create start-mcp.bat in the project root:

@echo off
cd /d "C:\path\to\qa-ai-agents"
node --loader ts-node/esm src/server.ts

Replace C:\path\to\qa-ai-agents with your actual project path.

Step 2 — Register the MCP server

claude mcp add -s user qa-ai-agents "C:\path\to\qa-ai-agents\start-mcp.bat"

Step 3 — Verify connection

# List all MCP servers and their status
claude mcp list

# Check your server specifically
claude mcp get qa-ai-agents

Expected output:

qa-ai-agents:
  Scope: User config (available in all your projects)
  Status: ✔ Connected
  Type: stdio
  Command: C:\path\to\qa-ai-agents\start-mcp.bat

Step 4 — Remove the server (if needed)

claude mcp remove qa-ai-agents -s user

---

Switching between Ollama and Claude API

Ollama → Claude API

Step 1 — Update .env:

USE_OLLAMA=false
ANTHROPIC_API_KEY=sk-ant-your-actual-key-here

Step 2 — Restart the MCP server to pick up new env:

claude mcp remove qa-ai-agents -s user
claude mcp add -s user qa-ai-agents "C:\path\to\qa-ai-agents\start-mcp.bat"

Step 3 — Verify correct provider is loaded:

npm run agent1

You should see:

Using Claude API: claude-sonnet-4-6

---

Claude API → Ollama

Step 1 — Make sure Ollama is running and model is pulled:

ollama list
# Should show qwen2.5-coder:7b or your preferred model

If model is not pulled yet:

ollama pull qwen2.5-coder:7b

Step 2 — Update .env:

USE_OLLAMA=true
OLLAMA_MODEL=qwen2.5-coder:7b

Step 3 — Restart the MCP server:

claude mcp remove qa-ai-agents -s user
claude mcp add -s user qa-ai-agents "C:\path\to\qa-ai-agents\start-mcp.bat"

Step 4 — Verify correct provider is loaded:

npm run agent1

You should see:

Using Ollama local model: qwen2.5-coder:7b

---

Provider comparison

	Ollama (local)	Claude API (cloud)
Cost	Free	~$0.18 per pipeline run
Speed	15–25 minutes	45–90 seconds
Quality	Good	Best
Internet required	No	Yes
Best for	Development and debugging	Production runs and demos

Tip: Use USE_OLLAMA=true while writing and debugging agent code to avoid burning API credits. Switch to USE_OLLAMA=false for actual pipeline runs and portfolio demos.

---

Environment variables

Variable	Required	Description
ANTHROPIC_API_KEY	Yes (if USE_OLLAMA=false)	Your Anthropic API key from console.anthropic.com
USE_OLLAMA	Yes	true for local Ollama, false for Claude API
OLLAMA_MODEL	No	Ollama model name. Default: qwen2.5-coder:7b