Google News Scraper
БесплатноНе проверенScrape Google News articles by keyword. Real-time brand monitoring, PR tracking, competitive intelligence. RSS + browser fallback. Node.js + Puppeteer.
Описание
Scrape Google News articles by keyword. Real-time brand monitoring, PR tracking, competitive intelligence. RSS + browser fallback. Node.js + Puppeteer.
README
Scrape Google News articles by keyword in real time. Monitor brand mentions, track PR coverage, and gather competitive intelligence from 50,000+ news sources worldwide. Built with Node.js, Puppeteer, and the Apify platform.
Run on Apify Available on RapidAPI License: ISC
What It Does
This Google News scraper extracts articles from Google News using a triple-fallback architecture:
- RSS Feed (fastest, most reliable) — Parses Google News RSS/XML feeds
- Google News Website (browser-based) — Scrapes the news.google.com interface directly
- Google Search News Tab (deepest) — Falls back to Google's main search news results
If one approach fails (CAPTCHA, rate limit), it automatically tries the next. You get results even when Google blocks standard methods.
What Data You Get
{
"keyword": "Tesla",
"title": "Tesla Announces New Gigafactory in Southeast Asia",
"source": "Reuters",
"publishedAt": "2026-03-10T14:30:00.000Z",
"snippet": "Tesla Inc said on Monday it would build a new manufacturing facility...",
"articleUrl": "https://www.reuters.com/business/autos/tesla-new-gigafactory-2026-03-10/",
"imageUrl": "https://...",
"fullText": "Full article text extracted from source (optional)...",
"scrapedAt": "2026-03-10T15:00:00.000Z"
}
Quick Start
Using the Apify API (Recommended)
The fastest way to get started — no setup required:
cURL
curl "https://api.apify.com/v2/acts/george.the.developer~google-news-monitor/run-sync-get-dataset-items?token=YOUR_API_TOKEN" \
-X POST \
-d '{
"keywords": ["Tesla", "OpenAI"],
"timeRange": "past_day",
"maxArticlesPerKeyword": 20
}' \
-H 'Content-Type: application/json'
Node.js
import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: 'YOUR_API_TOKEN' });
const run = await client.actor('george.the.developer/google-news-monitor').call({
keywords: ['Tesla', 'OpenAI', 'artificial intelligence'],
timeRange: 'past_day',
maxArticlesPerKeyword: 20,
extractFullText: false,
language: 'en',
country: 'US',
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(`Found ${items.length} articles`);
items.forEach(article => {
console.log(`[${article.source}] ${article.title}`);
});
Python
from apify_client import ApifyClient
client = ApifyClient("YOUR_API_TOKEN")
run = client.actor("george.the.developer/google-news-monitor").call(run_input={
"keywords": ["Tesla", "OpenAI"],
"timeRange": "past_day",
"maxArticlesPerKeyword": 20,
"extractFullText": False,
"language": "en",
"country": "US",
})
for article in client.dataset(run["defaultDatasetId"]).iterate_items():
print(f"[{article['source']}] {article['title']}")
print(f" URL: {article['articleUrl']}")
Use Cases
- Brand Monitoring — Track mentions of your company, product, or competitors across 50,000+ news sources
- PR & Media Intelligence — Measure press coverage, identify trending stories about your brand
- Competitive Intelligence — Monitor competitor announcements, partnerships, and product launches
- Market Research — Track industry trends, regulatory changes, and market signals
- Content Curation — Aggregate news for newsletters, dashboards, or AI-powered analysis
- AI Training Data — Build datasets of news articles for NLP, sentiment analysis, or LLM fine-tuning
Input Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
keywords |
string[] | required | Keywords or brand names to search |
timeRange |
string | past_day |
anytime, past_hour, past_day, past_week, past_month |
maxArticlesPerKeyword |
number | 20 | Max articles per keyword (1-100) |
extractFullText |
boolean | false | Visit each article to extract full text |
language |
string | en |
Language code (en, es, fr, de, ja, etc.) |
country |
string | US |
Country code (US, GB, DE, FR, JP, etc.) |
proxyConfiguration |
object | — | Proxy settings (Apify Proxy recommended) |
How It Works (Architecture)
Input Keywords
│
▼
┌─────────────────────┐
│ Approach 1: RSS │──── Success? ──► Parse XML ──► Results
│ (news.google.com) │
└─────────────────────┘
│ Failed/CAPTCHA
▼
┌─────────────────────┐
│ Approach 2: Web │──── Success? ──► Parse DOM ──► Results
│ (news.google.com) │
└─────────────────────┘
│ Failed/CAPTCHA
▼
┌─────────────────────┐
│ Approach 3: Search │──── Success? ──► Parse DOM ──► Results
│ (google.com/news) │
└─────────────────────┘
│
▼
Deduplicate + Optional Full-Text Extraction
│
▼
Output to Dataset
Run on Apify
Run this actor on Apify — no setup, no infrastructure, pay only for what you use.
- Cost: ~$0.003 per article found
- Speed: 20-100 articles per minute depending on settings
- Reliability: Triple-fallback ensures results even when Google blocks standard methods
Also Available on RapidAPI
Prefer a standard REST API? This scraper is also available on RapidAPI with simple API key authentication:
- Free tier: 50 requests/month
- Pro: $19/month (1,000 requests)
- Ultra: $49/month (5,000 requests)
- Mega: $129/month (20,000 requests)
Limitations
- This tool does not bypass paywalled article content. Full-text extraction works on freely accessible articles only.
- Google may serve CAPTCHAs for high-volume requests. Using Apify Proxy significantly reduces this.
- Rate limiting applies — respect Google's terms of service.
Related Tools
- LinkedIn Employee Scraper — Extract employee data from any company
- YouTube Transcript Extractor — Get video transcripts for AI/RAG
- Website Contact Scraper — Find emails & contacts from any website
- US Tariff Lookup — Look up import duty rates & HS codes
License
ISC License. See LICENSE for details.
Built by george.the.developer on Apify.
from github.com/the-ai-entrepreneur-ai-hub/google-news-scraper
Установка Google News Scraper
У этого сервера нет опубликованного пакета — он собирается из исходников. Открой репозиторий и следуй инструкции в README.
▸ github.com/the-ai-entrepreneur-ai-hub/google-news-scraperFAQ
Google News Scraper MCP бесплатный?
Да, Google News Scraper MCP бесплатный — установка в пару кликов через Unyly без оплаты.
Нужен ли API-ключ для Google News Scraper?
Нет, Google News Scraper работает без API-ключей и переменных окружения.
Google News Scraper — hosted или self-hosted?
Self-hosted: сервер запускается локально на твоей машине командой из раздела установки.
Как установить Google News Scraper в Claude Desktop, Claude Code или Cursor?
Открой Google News Scraper на unyly.org, выбери вкладку своего клиента (Claude Desktop, Claude Code, Cursor) и нажми Install — конфиг сгенерируется автоматически, без правки JSON.
Похожие MCP
Playwright
Browser automation, scraping, screenshots
автор: MicrosoftPuppeteer
Browser automation and web scraping.
автор: modelcontextprotocolGarmin Connect
An MCP server for Garmin Connect that provides access to fitness activities, health statistics, and sleep data by routing requests through a headless browser to
автор: etweisbergHiggsfield Unlimited
MCP server for Higgsfield AI that enables unlimited-mode image, video, audio generation, uploads, and job management via multiple parallel accounts, using brows
автор: nukIeeropentabs-dev/opentabs
Plugin-based server and Chrome extension that gives AI agents access to web apps through your authenticated browser session, with a plugin SDK.
автор: opentabs-devhlydecker/ucsc-genome-mcp
Interact with the UCSC Genome Browser API to find genomes, chromosomes and more.
автор: hlydecker34892002/bilibili-mcp-js
Search Bilibili content, with LangChain integration examples.
автор: 34892002achiya-automation/safari-mcp
Native Safari browser automation without a Chrome dependency.
автор: achiya-automationagent-infra/mcp-server-browser
Browser automation using Puppeteer, supporting local and remote browser connections.
автор: bytedanceaparajithn/agent-scraper-mcp
Web scraping: clean content extraction, CSS-selector scraping, Playwright screenshots, link and metadata extraction, and Google search.
автор: aparajithnCompare Google News Scraper with
Не уверен что выбрать?
Найди свой стек за 60 секунд
Автор?
Embed-бейдж для README
Похожее
Все в категории browse
