RokuHarness Server

A Model Context Protocol (MCP) server for comprehensive Roku automated testing. RokuHarness combines Roku WebDriver and ECP (External Control Protocol) to provide both UI state verification and device control - enabling true acceptance criteria validation, not just remote control simulation.

Why WebDriver vs ECP?

Capability	ECP	WebDriver
Press remote buttons	✅	✅
Launch apps	✅	✅
Query UI elements	❌	✅
Verify text is on screen	❌	✅
Check element attributes	❌	✅
Get SceneGraph XML	❌	✅
Take screenshots	❌	✅
Validate acceptance criteria	❌	✅

Bottom line: ECP can press buttons, WebDriver can verify what happens.

Architecture

Your Tests (via any MCP client)
    ↓
RokuHarness MCP Server (this project)
    ↓ (WebDriver HTTP API)
Roku WebDriver Server (from Roku's repo)
    ↓ (ECP + Debug APIs)
Roku Device (your sideloaded channel)

Key Point: The Roku WebDriver Server uses BOTH ECP (for control) and Roku's debug APIs (for UI verification). This MCP server provides a unified interface to both capabilities.

Prerequisites

1. Roku WebDriver Server

You need to download and run Roku's official WebDriver server:

# Clone Roku's automated testing repo
git clone https://github.com/rokudev/automated-channel-testing.git
cd automated-channel-testing

# Build the WebDriver server (requires Go)
cd src
go build

# Run the server
./RokuWebDriver  # Linux/Mac
# or
RokuWebDriver.exe  # Windows

The server will start on http://localhost:9000 by default.

Download pre-built binaries: Check the automated-channel-testing/bin folder for pre-compiled executables.

2. Sideloaded Channel

WebDriver requires your channel to be sideloaded in developer mode:

1. Enable developer mode on your Roku: Settings → System → About → Press Home 5x, Up, Rewind 2x, Fast Forward 2x
2. Package your channel as a .zip file
3. Visit http://YOUR_ROKU_IP in a browser
4. Upload and install your channel

Important: WebDriver only works with:

• Sideloaded developer channels (app ID: dev)
• Channels packaged with your developer account on that specific device
• SceneGraph-based channels (not legacy BrightScript)

3. This MCP Server

npm install
npm run build

Installation & Setup

1. Build this MCP server

npm install
npm run build

2. Configure Your MCP Client

For Claude Desktop:

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "rokuharness": {
      "command": "node",
      "args": ["/absolute/path/to/rokuharness-mcp/build/index.js"]
    }
  }
}

For Custom MCP Clients:

RokuHarness is MCP-agnostic and works with any MCP client. See "Using Without Claude Desktop" section below for integration examples.

3. Start the Roku WebDriver Server

In a separate terminal:

cd /path/to/automated-channel-testing/bin
./RokuWebDriver  # or RokuWebDriver.exe on Windows

Keep this running while testing.

4. Restart Claude Desktop

The MCP server will now be available in Claude.

Usage Guide

Step 1: Create a Session

Every test session starts by creating a WebDriver session:

Create a Roku WebDriver session:
- WebDriver URL: http://localhost:9000
- Roku IP: 192.168.1.100
- App: dev

This connects to your sideloaded channel.

Step 2: Get UI Source (Critical!)

Before writing any verification queries, inspect the UI:

Get the UI source to see what elements are on screen

You'll get back SceneGraph XML like:

<Scene>
  <Label id="titleLabel" text="Welcome to My App" focused="false" />
  <Button id="loginButton" text="Log In" focused="true" />
  <Poster id="heroImage" uri="https://..." />
</Scene>

This XML shows you:

• Tags: Component types (Label, Button, Poster, etc.)
• Attributes: Properties like id, text, focused, visible
• Hierarchy: Nested structure

Step 3: Verify Elements

Now you can write acceptance criteria tests:

Verify that a Label with text "Welcome to My App" is present on screen

Verify that a Button with id "loginButton" and focused=true is present

Verify that the login button text says "Log In"

Step 4: Navigate and Test

Navigate using these keys: ["Down", "Down", "Select"]
Then verify that a Label with text "Account Settings" appears

Step 5: Run Complete Test Scenarios

Run this acceptance test:

Test: Login Flow
Steps:
1. Press Select to click login button
2. Verify keyboard screen appears (look for Label with text "Enter Email")
3. Send text "[email protected]"
4. Press Down to go to password field
5. Send text "password123"
6. Press Select to submit
7. Verify success message appears
8. Take screenshot

Available Tools

Session Management

create_webdriver_session

Creates a new WebDriver session.

Parameters:

• webdriver_url (optional): URL of WebDriver server (default: http://localhost:9000)
• roku_ip (required): IP address of Roku device
• app (optional): App ID or "dev" for sideloaded (default: "dev")

Example:

Create a WebDriver session for Roku at 192.168.1.100

end_webdriver_session

Ends the current session and cleans up.

---

UI State Verification (The Key Features!)

get_ui_source

Get the current UI hierarchy as XML or JSON.

Parameters:

• parsed (optional): Return JSON instead of XML

Example:

Get the UI source to see current screen structure

Returns:

<Scene>
  <LayoutGroup id="mainLayout">
    <Label id="title" text="Home Screen" />
    <Button id="playButton" text="Play" focused="true" />
  </LayoutGroup>
</Scene>

This is essential for understanding what elements exist and how to query them.

find_element

Search for a specific element on screen.

Parameters:

• text (optional): Text content to match
• tag (optional): SceneGraph component type
• attributes (optional): Attribute key-value pairs

Examples:

Find a Label with text "Home Screen"

Find a Button with id "playButton"

Find an element with tag "Poster" and attribute uri="https://example.com/image.jpg"

verify_element_present

Check if an element exists (returns true/false).

Parameters:

• Same as find_element
• timeout_ms (optional): How long to wait (default: 10000)

Examples:

Verify a Label with text "Loading..." is present

Check if login button is focused: Button with focused=true

verify_screen_loaded

Wait for a specific screen to fully load.

Parameters:

• screen_marker: Element query that identifies the screen
• timeout_ms (optional): Maximum wait time

Example:

Verify the home screen loaded by checking for Label with text "Featured Content"

---

Navigation & Input

press_key

Press a single remote button.

Parameters:

• key: Button name (Home, Back, Up, Down, Left, Right, Select, Play, Pause, etc.)

navigate

Execute a sequence of button presses.

Parameters:

• keys: Array of keys to press
• delay_ms (optional): Delay between presses

Example:

Navigate: Down, Down, Right, Select with 750ms delays

send_text

Send text input (for keyboards/forms).

Parameters:

• text: Text to type

---

Media & Apps

launch_app

Launch an app with optional deep linking.

Parameters:

• app_id: App ID ("dev" for sideloaded)
• content_id (optional): Deep link content ID
• media_type (optional): Type (movie, series, etc.)

get_player_state

Get current playback state.

Returns: Position, duration, state, buffering info

get_installed_apps

List all installed apps.

---

Screenshots

take_screenshot

Capture current screen.

Parameters:

• save_path (optional): Where to save the image

Example:

Take a screenshot and save to /tmp/login_screen.png

---

Acceptance Testing

run_acceptance_test

Execute a complete test case with multiple steps.

Parameters:

• test_name: Name of the test
• steps: Array of test steps

Step types:

• navigate: Execute key sequence
• verify_element: Check element is present
• press_key: Press single key
• send_text: Type text
• wait: Pause for duration
• screenshot: Capture screen

Example:

Run this acceptance test:

Name: "Video Playback Test"

Steps:
1. Action: navigate, Keys: ["Down", "Down", "Select"], Description: "Select first video"
2. Action: verify_element, Query: {tag: "Video", attributes: {state: "playing"}}, Description: "Verify video is playing"
3. Action: wait, Duration: 5000, Description: "Let video play for 5 seconds"
4. Action: press_key, Key: "Pause", Description: "Pause playback"
5. Action: verify_element, Query: {tag: "Video", attributes: {state: "paused"}}, Description: "Verify video paused"
6. Action: screenshot, Description: "Capture paused state"

---

Element Query Syntax

Elements are queried using combinations of:

By Text

{ "text": "Log In" }

Finds elements containing this exact text.

By Tag

{ "tag": "Button" }

Finds elements of this SceneGraph type.

Common tags:

• Label - Text display
• Button - Interactive button
• Poster - Image
• Video - Video player
• LayoutGroup - Container
• RowList - Scrollable list
• Grid - Grid layout

By Attributes

{
  "attributes": {
    "id": "loginButton",
    "focused": "true"
  }
}

Common attributes:

• id - Unique identifier
• focused - Has focus (true/false)
• visible - Is visible (true/false)
• text - Text content
• uri - Image/video URI

Combined Queries

{
  "tag": "Button",
  "text": "Log In",
  "attributes": {
    "focused": "true"
  }
}

Finds a Button with text "Log In" that currently has focus.

---

Real-World Examples

Example 1: Validate Login Screen

1. Create WebDriver session for Roku at 192.168.1.100

2. Get UI source to inspect elements

3. Verify these elements are present:
   - Label with text "Sign In"
   - Button with text "Email Login"
   - Button with text "Guest Mode"

4. Take screenshot for documentation

Example 2: Test Video Playback

Run this acceptance test:

Name: "Video Playback Verification"

Steps:
1. Navigate to content: ["Down", "Down", "Select"]
2. Verify video player loaded: tag=Video
3. Wait 3 seconds for playback to start
4. Verify video is playing: tag=Video, attributes={state: "playing"}
5. Press "Info" to show controls
6. Verify play/pause button visible: tag=Button, text="Pause"
7. Take screenshot of player controls

Example 3: Search Functionality

Test search feature:

1. Press "Search" key
2. Verify keyboard screen: Label with text "Search"
3. Send text "Breaking Bad"
4. Press "Select" to submit
5. Verify results loaded: Label with text "Results for: Breaking Bad"
6. Verify at least one result: tag=Poster (poster images indicate results)

Example 4: Settings Navigation

Navigate to settings and verify:

1. Press Home
2. Navigate: ["Down", "Down", "Down", "Right", "Right", "Select"]
3. Verify settings screen: Label with text "Settings"
4. Navigate: ["Down", "Select"]
5. Verify account screen: Label with text "Account Information"
6. Get UI source to document screen structure

---

Troubleshooting

"No active session"

You must call create_webdriver_session before any other commands.

"WebDriver server not responding"

Ensure the Roku WebDriver server is running:

./RokuWebDriver

"Element not found"

1. Get the UI source first: get_ui_source
2. Inspect the actual XML structure
3. Adjust your query to match actual elements
4. Check spelling and capitalization (XML is case-sensitive)

"Cannot get source from channel"

• Only works with sideloaded channels or channels packaged on that device
• Production channels block source access (security feature)
• Make sure your channel is running (not on Home screen)

Screenshots not working

Screenshots only work when:

• Your sideloaded channel is active
• Developer mode is enabled
• WebDriver has proper access

Slow queries

• WebDriver queries can take 500ms-2s depending on complexity
• Use specific queries (tag + attributes) for faster results
• Avoid overly broad queries

---

Roku WebDriver Limitations

1. Sideloaded channels only - Production channels block UI introspection
2. SceneGraph only - Legacy BrightScript channels not supported
3. No visual comparisons - You get XML structure, not rendered pixels
4. Element bounds are relative - Absolute position checking is complex
5. No direct element interaction - You still navigate with D-pad, not clicks

---

Best Practices

1. Always Inspect First

Get UI source → Understand structure → Write queries

2. Use Specific Queries

❌ { "text": "Play" }  // Might match multiple elements
✅ { "tag": "Button", "id": "mainPlayButton", "text": "Play" }

3. Wait for Screens to Load

verify_screen_loaded with appropriate timeout

4. Test One Thing at a Time

Break complex flows into individual test cases.

5. Take Screenshots

Document state before/after critical steps.

6. Use Acceptance Test Tool

For multi-step scenarios, use run_acceptance_test to get structured results.

---

Integration with CI/CD

This MCP server can be integrated into CI/CD pipelines:

1. Provision Roku devices in your test lab
2. Start WebDriver server on each
3. Run tests via MCP server
4. Collect results and screenshots
5. Publish test reports

---

Using Without Claude Desktop

RokuHarness is built on the open MCP standard and works with any MCP client. Here are integration examples:

Python CI/CD Integration

from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client

# Connect to RokuHarness MCP server
server_params = StdioServerParameters(
    command="node",
    args=["/path/to/rokuharness-mcp/build/index.js"]
)

async with stdio_client(server_params) as (read, write):
    async with ClientSession(read, write) as session:
        await session.initialize()
        
        # Create WebDriver session
        result = await session.call_tool(
            "create_webdriver_session",
            arguments={
                "roku_ip": "192.168.1.100",
                "app": "dev"
            }
        )
        
        # Get UI source
        ui_source = await session.call_tool("get_ui_source", {})
        
        # Verify element
        verify_result = await session.call_tool(
            "verify_element_present",
            arguments={
                "text": "Welcome",
                "tag": "Label"
            }
        )
        
        # Assert in your test framework
        assert verify_result["present"] == True
        
        # Run acceptance test
        test_result = await session.call_tool(
            "run_acceptance_test",
            arguments={
                "test_name": "Login Flow",
                "steps": [
                    {
                        "action": "navigate",
                        "description": "Go to login",
                        "keys": ["Down", "Down", "Select"]
                    },
                    {
                        "action": "verify_element",
                        "description": "Check login screen",
                        "element_query": {"text": "Sign In"}
                    }
                ]
            }
        )
        
        print(f"Test Status: {test_result['summary']['status']}")

Node.js Integration

import { Client } from '@modelcontextprotocol/sdk/client/index.js';
import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js';

const transport = new StdioClientTransport({
  command: 'node',
  args: ['/path/to/rokuharness-mcp/build/index.js']
});

const client = new Client({
  name: 'roku-test-runner',
  version: '1.0.0'
}, {
  capabilities: {}
});

await client.connect(transport);

// Create session
const session = await client.request({
  method: 'tools/call',
  params: {
    name: 'create_webdriver_session',
    arguments: {
      roku_ip: '192.168.1.100',
      app: 'dev'
    }
  }
});

// Verify element
const result = await client.request({
  method: 'tools/call',
  params: {
    name: 'verify_element_present',
    arguments: {
      tag: 'Button',
      text: 'Play'
    }
  }
});

console.log('Element present:', result.present);

Integration Points

✅ CI/CD Pipelines - Jenkins, GitHub Actions, GitLab CI, CircleCI
✅ Test Frameworks - Jest, Mocha, Pytest, JUnit
✅ QA Platforms - TestRail, Zephyr, qTest, Xray
✅ Custom Dashboards - Build your own test runner UI
✅ Scheduled Testing - Cron jobs, AWS Lambda, Azure Functions
✅ Any MCP-compatible tool - The protocol is completely open

Example GitHub Actions Workflow

name: Roku UI Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@v3
      
      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '20'
      
      - name: Install dependencies
        run: |
          cd rokuharness-mcp
          npm install
          npm run build
      
      - name: Start Roku WebDriver Server
        run: |
          wget https://github.com/rokudev/automated-channel-testing/releases/download/v1.0/RokuWebDriver
          chmod +x RokuWebDriver
          ./RokuWebDriver &
          
      - name: Run tests
        run: python tests/run_roku_tests.py
        env:
          ROKU_IP: ${{ secrets.ROKU_IP }}
          
      - name: Upload screenshots
        if: always()
        uses: actions/upload-artifact@v3
        with:
          name: test-screenshots
          path: screenshots/

---

Comparison with Other Tools

vs Roku Robot Framework

• This: Natural language via Claude, MCP protocol
• Robot: Keyword-driven, separate test files

vs Appium Roku Driver

• This: Direct WebDriver access, simpler setup
• Appium: Appium ecosystem integration, more tooling

vs Manual Testing

• This: Automated, repeatable, fast
• Manual: Comprehensive but slow, expensive

---

Resources

• Roku WebDriver Documentation
• Automated Channel Testing Repo
• SceneGraph XML Reference
• Roku Developer Portal

---

Support

For issues or questions:

• Roku WebDriver: Roku Community Forums
• This MCP Server: Open an issue in this repository
• Claude/MCP: Anthropic Documentation

---

License

MIT License - Free to use for your Roku testing needs.