Web Api Scrapper

(ships as the browser-flow-tracker package on npm)

npm version license: MIT website

🌐 Website: apiflowtracker.com — one-line install & quick start.

A local MCP server for Claude & Cursor that records the API flow of any web page — as you click through it — and turns it into documentation.

Just say to your AI:

🗣️ "let's record the session for this url https://your-app.com"

A real browser window opens → you navigate the flow (log in, click around) → say "done" or close the window → you get a clean Markdown flow doc (with a sequence diagram), a structured .flow.json, and a standard .har — noise filtered, secrets redacted.

⚡ Quick start (MCP)

1. Install — one line, no Node.js required:

# macOS & Linux (Terminal)
curl -fsSL https://apiflowtracker.com/install.sh | sh

# Windows (PowerShell)
irm https://apiflowtracker.com/install.ps1 | iex

2. Add to Claude Code / Cursor (the installer prints your exact path):

{
  "mcpServers": {
    "browser-flow-tracker": {
      "command": "/usr/local/bin/browser-flow-tracker"
    }
  }
}

3. Restart your AI app and say the phrase. That's the whole setup — details, npx alternative, and where the config lives are in Connecting to Claude / Cursor.

---

"What is this website actually doing behind the scenes?"

You know that feeling when you click a button on a website and a bunch of invisible stuff happens — APIs get called, magic occurs — but you have no idea what's going on unless you open the scary black DevTools console and squint at a wall of text?

This tool is your spy. 🔍 It watches the page while you click, notes down every API conversation in order, and hands you a readable document with a diagram. No more squinting. No more console. (Prefer the terminal? There's a CLI too.)

---

🎯 What you get out of it

After you record a flow, the tool drops 3 files in a folder for you:

File	Emoji	What it's for	Who reads it
something.md	📄	A nice, readable document: overview, a diagram, a table of every API call in order, and the details	You! (and Claude/Cursor)
something.flow.json	🤖	The same info in "robot language"	Claude / Cursor, to help you write even better docs
something.har	🔧	An industry-standard file	Developers, tools like Postman/DevTools

👉 The one you'll actually open and read is the .md file. Open it in any Markdown viewer, VS Code, or even just a text editor.

📁 Where do the files go?

By default, everything is saved into a folder called recordings/ right inside this project:

Scrapper/
└── recordings/
    ├── flow-2026-07-01T14-30-00.md      ← 📄 open this one
    ├── flow-2026-07-01T14-30-00.flow.json
    └── flow-2026-07-01T14-30-00.har

Each recording is automatically named with the date and time so they never overwrite each other. Want a nicer name or a different folder? See the options below.

---

🍬 What a finished document looks like

Here's a taste of what ends up in your .md file:

Sequence diagram

sequenceDiagram
    participant B as Browser
    participant S0 as api.myapp.com
    B->>S0: GET /products
    S0-->>B: 200
    B->>S0: POST /cart/add
    S0-->>B: 201

Call sequence

#	Method	Endpoint	Status	Time
1	GET	api.myapp.com/products	200	120ms
2	POST	api.myapp.com/cart/add	201	340ms

…plus the full request and response details for each call. 🎉

It's smart, too: it throws away the boring noise (images, fonts, styling, tracking/analytics pixels) and keeps only the meaningful stuff. It hides your secrets by default — auth headers and cookies, plus password/token-style fields inside request & response bodies and URLs — so you can share the doc safely. And if your flow opens a popup or a new tab (an OAuth login, a payment window), that tab's API calls are captured too, in the same recording.

---

🚀 First-time setup (do this once)

✅ Using it through Claude or Cursor? You can skip this whole section — just add the tiny config in Connecting to Claude / Cursor and npx handles the rest. The steps below are only for the Terminal / bft command usage.

You need this one-time setup. It's two steps.

1. Make sure you have Node.js. Open the Terminal app and paste this:

node --version

If you see a number like v18 or higher, you're good. If it says "command not found", download Node from nodejs.org and install it (just click Next → Next → Done).

2. Install the tool's bits. Paste this into Terminal:

cd path/to/browser-flow-tracker   # 👈 the folder where this tool lives
npm install

Wait a few seconds. Done forever. ✅

💡 What's "Terminal"? It's the app on your Mac where you type commands. Press Cmd + Space, type "Terminal", hit Enter.

📌 What do I put instead of path/to/browser-flow-tracker? The easiest trick: type cd (with a space), then drag the tool's folder from Finder into the Terminal window and hit Enter — the correct path fills itself in. From then on, the cd ... line is the same in every command below.

---

🎬 How to use it — 3 ways

Pick whichever feels comfortable. They all produce the same nice files.

Way 1: 🗣️ Just ask Claude or Cursor (easiest — zero commands)

Once the tool is connected (it already is — see Connecting to Claude/Cursor), you literally just say the magic phrase:

"Let's record the session for this url https://your-website.com"

Here's what happens automatically:

1. 🪟 The AI opens a real browser window at that URL (it picks a free port itself — nothing for you to configure).
2. 🖱️ You click through the flow you want documented. You drive — the AI does not click for you.
3. 🛑 When you're finished, either say "done" or just close the browser window.
4. 📄 The AI writes the .flow.json / .har / .md files and hands you a clean doc of the APIs it saw.

You don't type a single command. ✨

🔐 Logging in? The window uses its own saved profile, so you only log in the first time — future recordings remember you.

⚠️ If your AI ignores the phrase and starts poking around on its own instead of using the tool, you haven't installed the rule yet — see Make your AI always use it.

---

Way 2: 🧼 Let the tool open a browser for you (Terminal)

Paste this into Terminal (change the website):

cd path/to/browser-flow-tracker   # 👈 the folder where this tool lives
node bin/bft.js record --launch --browser brave --url https://your-website.com

A browser window pops open on its own private profile (it never touches your normal Brave). Click around, do the thing you want to document — logging in if you need to. When you're done, either press Ctrl + C in Terminal or just close the browser window — both save the recording.

💥 Your files appear in the recordings/ folder.

🔐 You only log in the first time — the private profile remembers you for next time.

Don't have Brave? Run node bin/bft.js list to see which browsers you have, then swap brave for arc, chrome, etc.

---

Way 3: 🔐 Watch your own main browser (advanced)

Usually Way 1 or Way 2 is enough even for logged-in sites (you log in once in the tool's own window and it's remembered). Use this only if you specifically need to record inside your existing, already-open browser session.

Step 1. Ask the tool for the magic command:

cd path/to/browser-flow-tracker   # 👈 the folder where this tool lives
node bin/bft.js attach-help brave

It prints an exact command — copy it, quit Brave completely, and paste that command into Terminal. Brave reopens in "watch me" mode.

Step 2. Log in and go to where the flow starts. Then, in a new Terminal window:

cd path/to/browser-flow-tracker   # 👈 the folder where this tool lives
node bin/bft.js record --attach --url-match your-website.com

Step 3. Do your thing in the browser, then press Ctrl + C in Terminal.

💥 Files appear in recordings/.

---

🎛️ Handy options

Add any of these to the end of a record command:

Option	What it does	Example
--name checkout	Prefix the files with checkout (a timestamp is still appended so recordings never overwrite each other)	--name checkout
--out ~/Desktop/docs	Save files somewhere else (like your Desktop)	--out ~/Desktop/docs
--title "Checkout Flow"	Give the document a proper title	--title "Checkout Flow"
--url-match myapp	Watch the correct browser tab	--url-match myapp
--scope myapp.com	Declare which domains the flow is about; calls to other hosts are listed separately as third-party (default: auto-detected from the pages you visit)	--scope myapp.com,api.myapp.io
--no-events	Skip the analytics events timeline (GA4/Segment/Mixpanel events are otherwise decoded into .events.json + .events.md)	--no-events
--include-noise	Keep everything (images, fonts, tracking too)	--include-noise
--no-redact	Show passwords/tokens (⚠️ don't share the file then)	--no-redact

Full example:

node bin/bft.js record --launch --browser brave \
  --url https://shop.example.com \
  --name checkout-flow \
  --title "Shop Checkout Flow" \
  --out ~/Desktop

---

🤖 Connecting to Claude / Cursor

✅ Recommended: one-line install (no Node, works on macOS, Windows & Linux)

The most reliable way — no Node.js required — and it avoids the spawn npx ENOENT PATH problems that trip up GUI apps. It installs a standalone binary and prints the exact MCP config to paste.

macOS & Linux — in Terminal:

curl -fsSL https://apiflowtracker.com/install.sh | sh

Then your MCP config command is (same path on every Mac/Linux machine):

{ "mcpServers": { "browser-flow-tracker": { "command": "/usr/local/bin/browser-flow-tracker" } } }

Windows — in PowerShell:

irm https://apiflowtracker.com/install.ps1 | iex

Then your MCP config command is the path the installer prints, e.g.:

{ "mcpServers": { "browser-flow-tracker": { "command": "C:\\Users\\you\\AppData\\Local\\browser-flow-tracker\\browser-flow-tracker.exe" } } }

The installer downloads the right build for your CPU automatically (Apple Silicon or Intel Mac, x64 or arm64 Linux, x64 Windows). Restart your AI app after adding the config.

Alternative: via npx (only if you already have Node.js)

If Node 18+ is installed and on your app's PATH, you can skip the install and run it straight from npm:

{
  "mcpServers": {
    "browser-flow-tracker": {
      "command": "npx",
      "args": ["-y", "browser-flow-tracker@latest"]
    }
  }
}

Claude Code shortcut: claude mcp add browser-flow-tracker -- npx -y browser-flow-tracker@latest

If you hit spawn npx ENOENT, your app can't see npx on its PATH — use the one-line installer above instead; it sidesteps the whole issue.

Where the config file lives

• Claude Code: .mcp.json in a project, or your user config (managed by claude mcp).
• Cursor: .cursor/mcp.json (per project) or ~/.cursor/mcp.json (global).

⚠️ Already using other MCP tools? Don't replace your file — add the browser-flow-tracker block inside your existing mcpServers, with a comma after your previous tool. Keep it valid JSON (commas between entries, none after the last).

Ready-made copies are in this repo as .mcp.json.example and .cursor/mcp.json.example.

Prefer to run from source? (for the CLI or development)

You don't need this for the AI use case, but if you want the bft command-line tool or want to hack on the code:

git clone https://github.com/devggaurav/web-Api-scrapper.git
cd web-Api-scrapper
npm install
npm test   # unit tests (redaction, filtering, exporters)
# MCP: point your config's command at "node" with args ["<full-path>/mcp/server.js"]
# CLI: node bin/bft.js record --launch --browser brave --url https://example.com

---

Behind the scenes it has four skills it uses automatically:

Skill	What it does
list_browsers	See which browsers you have
start_tracking	Start watching
get_flow	Peek at what's been captured so far (without stopping)
stop_tracking	Stop and save the files

---

🧲 Make your AI always use it

Sometimes an AI will "helpfully" try to analyze a page its own way (poking around with scripts) instead of using this tool. To stop that, give it a standing rule so the phrase "let's record the session for this url …" always triggers the tool. Do it once and it works in every project.

Cursor

Add a User Rule (applies in all projects): Settings → Rules & Memories → User Rules → + Add, and paste:

When I say "let's record the session for this url <URL>" — or ask you to record/
analyze/document the API or network flow of a page — ALWAYS use the browser-flow-
tracker MCP tools (start_tracking, get_flow, stop_tracking). Do not launch browsers,
touch debug ports, or sniff traffic yourself.
Loop: call start_tracking with launch:true, browser:"brave", url:<URL>. Then let ME
navigate — do not click for me. The recording ends when I say "done" (call
stop_tracking with a name+title) OR I close the browser (it auto-finalizes; detect via
get_flow returning status:"ended"). Both write .flow.json/.har/.md — read the
.flow.json and write me a clean flow doc.

(A copy of this rule also ships in .cursor/rules/browser-flow-tracker.mdc, which applies automatically whenever you work inside this project.)

Claude Code

Add the same guidance to your global ~/.claude/CLAUDE.md (create the file if it doesn't exist) so it applies in every project. The exact text is in .cursor/rules/browser-flow-tracker.mdc — copy the body into ~/.claude/CLAUDE.md.

After adding a rule, restart the AI app so it picks it up.

---

❓ Common questions & fixes

"My AI didn't use the tool — it started doing its own thing." You haven't given it the standing rule yet. See Make your AI always use it, then restart the AI app.

"How do I stop a recording?" Two ways, whichever you like: say "done" to your AI, or just close the browser window. Both save the files automatically. (In Terminal, Ctrl + C also works.)

"Do I have to log in every time?" No — the tool's browser window keeps its own saved profile, so you log in the first time and it's remembered afterwards.

"It says 'No known browsers found'." Run node bin/bft.js list. If it's empty, you don't have a supported browser installed. Install Brave or Chrome (both free) and try again.

"Nothing showed up in my recording." Make sure you actually did the thing in the browser before you stopped. Some pages need a click or a page load to fire their APIs.

"Where are my files again?" In the recordings/ folder — unless you used --out (or the AI set a different location). The tool prints the exact file paths when it finishes, and the AI can read them back to you.

"Can I use Safari?" Not yet 😔 — Safari and Firefox work differently under the hood. For now use Brave, Chrome, Arc, or Edge. (Safari support is on the roadmap.)

"Is it safe to share the document?" By default, yes. The tool redacts secrets in all three places they show up: auth/cookie headers, token-style URL parameters (?token=…, ?api_key=…), and password/token-style fields inside JSON & form request/response bodies (password, access_token, client_secret, otp, card numbers, …). Field names it doesn't recognize as secrets are kept — so if an app uses an unusual name for a secret, skim the doc once before sharing. Only if you used --no-redact should you be careful.

"My flow opens a popup / new tab — is that captured?" Yes. OAuth windows, payment popups, and "open in new tab" links are attached to automatically (they're even paused for a split second at birth so their very first request isn't missed). Their calls appear in the same document, marked with the tab they came from.

---

🧠 How it works (the 20-second version)

The tool speaks the same secret language browsers use to talk to their own developer tools (it's called the "Chrome DevTools Protocol"). So it can listen in on every network request a page makes — the same info you'd see in that scary DevTools tab — and then it cleans it up, filters out the junk, and formats it into something a human can actually read. That's it. 🎩

---

🗺️ Roadmap (coming later)

• 🧭 Safari & Firefox support (via a different listening method)
• 🔀 "Diff" mode — compare two recordings to see what changed
• 🧩 Auto-group calls into logical steps in the doc

---

Made to save you from ever having to open the browser console again. Happy tracking! 🕵️‍♀️

🌐 apiflowtracker.com · 📦 npm · 🐙 GitHub