Command Palette

Search for a command to run...

UnylyUnyly
Весь каталог

Google News Scraper

БесплатноНе проверен

Scrape Google News articles by keyword. Real-time brand monitoring, PR tracking, competitive intelligence. RSS + browser fallback. Node.js + Puppeteer.

GitHubEmbed

Описание

Scrape Google News articles by keyword. Real-time brand monitoring, PR tracking, competitive intelligence. RSS + browser fallback. Node.js + Puppeteer.

README

Scrape Google News articles by keyword in real time. Monitor brand mentions, track PR coverage, and gather competitive intelligence from 50,000+ news sources worldwide. Built with Node.js, Puppeteer, and the Apify platform.

Run on Apify Available on RapidAPI License: ISC

What It Does

This Google News scraper extracts articles from Google News using a triple-fallback architecture:

  1. RSS Feed (fastest, most reliable) — Parses Google News RSS/XML feeds
  2. Google News Website (browser-based) — Scrapes the news.google.com interface directly
  3. Google Search News Tab (deepest) — Falls back to Google's main search news results

If one approach fails (CAPTCHA, rate limit), it automatically tries the next. You get results even when Google blocks standard methods.

What Data You Get

{
  "keyword": "Tesla",
  "title": "Tesla Announces New Gigafactory in Southeast Asia",
  "source": "Reuters",
  "publishedAt": "2026-03-10T14:30:00.000Z",
  "snippet": "Tesla Inc said on Monday it would build a new manufacturing facility...",
  "articleUrl": "https://www.reuters.com/business/autos/tesla-new-gigafactory-2026-03-10/",
  "imageUrl": "https://...",
  "fullText": "Full article text extracted from source (optional)...",
  "scrapedAt": "2026-03-10T15:00:00.000Z"
}

Quick Start

Using the Apify API (Recommended)

The fastest way to get started — no setup required:

cURL

curl "https://api.apify.com/v2/acts/george.the.developer~google-news-monitor/run-sync-get-dataset-items?token=YOUR_API_TOKEN" \
  -X POST \
  -d '{
    "keywords": ["Tesla", "OpenAI"],
    "timeRange": "past_day",
    "maxArticlesPerKeyword": 20
  }' \
  -H 'Content-Type: application/json'

Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_API_TOKEN' });

const run = await client.actor('george.the.developer/google-news-monitor').call({
    keywords: ['Tesla', 'OpenAI', 'artificial intelligence'],
    timeRange: 'past_day',
    maxArticlesPerKeyword: 20,
    extractFullText: false,
    language: 'en',
    country: 'US',
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(`Found ${items.length} articles`);
items.forEach(article => {
    console.log(`[${article.source}] ${article.title}`);
});

Python

from apify_client import ApifyClient

client = ApifyClient("YOUR_API_TOKEN")

run = client.actor("george.the.developer/google-news-monitor").call(run_input={
    "keywords": ["Tesla", "OpenAI"],
    "timeRange": "past_day",
    "maxArticlesPerKeyword": 20,
    "extractFullText": False,
    "language": "en",
    "country": "US",
})

for article in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"[{article['source']}] {article['title']}")
    print(f"  URL: {article['articleUrl']}")

Use Cases

  • Brand Monitoring — Track mentions of your company, product, or competitors across 50,000+ news sources
  • PR & Media Intelligence — Measure press coverage, identify trending stories about your brand
  • Competitive Intelligence — Monitor competitor announcements, partnerships, and product launches
  • Market Research — Track industry trends, regulatory changes, and market signals
  • Content Curation — Aggregate news for newsletters, dashboards, or AI-powered analysis
  • AI Training Data — Build datasets of news articles for NLP, sentiment analysis, or LLM fine-tuning

Input Parameters

Parameter Type Default Description
keywords string[] required Keywords or brand names to search
timeRange string past_day anytime, past_hour, past_day, past_week, past_month
maxArticlesPerKeyword number 20 Max articles per keyword (1-100)
extractFullText boolean false Visit each article to extract full text
language string en Language code (en, es, fr, de, ja, etc.)
country string US Country code (US, GB, DE, FR, JP, etc.)
proxyConfiguration object — Proxy settings (Apify Proxy recommended)

How It Works (Architecture)

Input Keywords
     │
     ▼
┌─────────────────────┐
│  Approach 1: RSS    │──── Success? ──► Parse XML ──► Results
│  (news.google.com)  │
└─────────────────────┘
     │ Failed/CAPTCHA
     ▼
┌─────────────────────┐
│  Approach 2: Web    │──── Success? ──► Parse DOM ──► Results
│  (news.google.com)  │
└─────────────────────┘
     │ Failed/CAPTCHA
     ▼
┌─────────────────────┐
│  Approach 3: Search │──── Success? ──► Parse DOM ──► Results
│  (google.com/news)  │
└─────────────────────┘
     │
     ▼
  Deduplicate + Optional Full-Text Extraction
     │
     ▼
  Output to Dataset

Run on Apify

Run this actor on Apify — no setup, no infrastructure, pay only for what you use.

  • Cost: ~$0.003 per article found
  • Speed: 20-100 articles per minute depending on settings
  • Reliability: Triple-fallback ensures results even when Google blocks standard methods

Also Available on RapidAPI

Prefer a standard REST API? This scraper is also available on RapidAPI with simple API key authentication:

  • Free tier: 50 requests/month
  • Pro: $19/month (1,000 requests)
  • Ultra: $49/month (5,000 requests)
  • Mega: $129/month (20,000 requests)

Limitations

  • This tool does not bypass paywalled article content. Full-text extraction works on freely accessible articles only.
  • Google may serve CAPTCHAs for high-volume requests. Using Apify Proxy significantly reduces this.
  • Rate limiting applies — respect Google's terms of service.

Related Tools

License

ISC License. See LICENSE for details.


Built by george.the.developer on Apify.

from github.com/the-ai-entrepreneur-ai-hub/google-news-scraper

Установка Google News Scraper

У этого сервера нет опубликованного пакета — он собирается из исходников. Открой репозиторий и следуй инструкции в README.

▸ github.com/the-ai-entrepreneur-ai-hub/google-news-scraper

FAQ

Google News Scraper MCP бесплатный?

Да, Google News Scraper MCP бесплатный — установка в пару кликов через Unyly без оплаты.

Нужен ли API-ключ для Google News Scraper?

Нет, Google News Scraper работает без API-ключей и переменных окружения.

Google News Scraper — hosted или self-hosted?

Self-hosted: сервер запускается локально на твоей машине командой из раздела установки.

Как установить Google News Scraper в Claude Desktop, Claude Code или Cursor?

Открой Google News Scraper на unyly.org, выбери вкладку своего клиента (Claude Desktop, Claude Code, Cursor) и нажми Install — конфиг сгенерируется автоматически, без правки JSON.

Похожие MCP

Compare Google News Scraper with

Не уверен что выбрать?

Найди свой стек за 60 секунд

Автор?

Embed-бейдж для README

Похожее

Все в категории browse