The official YouTube Data API gives you 10,000 quota units a day — one search can eat 100 of them. For real trend monitoring or comment analysis you need a different pipeline. Here's the scraper route.
🤖 Try it live — free Poe bots that generate the exact pipeline for your use case: Lead List Generator · Job Postings Miner · Amazon Product Miner · SERP Rank Tracker · Social Lead Finder
The stack
- Apify — YouTube actors for search, channels, comments, transcripts. Free monthly credits.
- Python — feed results straight into pandas or your DB.
Step 1 — Token
Create an Apify account → Settings → Integrations → API token.
Step 2 — Search + channel data
curl -X POST "https://api.apify.com/v2/acts/streamers~youtube-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{"searchQueries": ["ai agent tutorial"], "maxResults": 50, "sortingOrder": "date"}' Fields per video: title, channelName, viewCount, likes, date, duration, url, thumbnailUrl.
Step 3 — Comments = market research
from apify_client import ApifyClient client = ApifyClient("APIFY_TOKEN") run = client.actor("bernardo/youtube-scraper").call(run_input={ "startUrls": [{"url": "https://www.youtube.com/watch?v=VIDEO_ID"}], "maxComments": 200, }) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): for c in item.get("comments", [])[:200]: text = c.get("comment", "") if any(w in text.lower() for w in ["how do", "doesn't work", "wish", "alternatives"]): print("PAIN:", text[:160]) Comments complaining = product gaps. Comments asking "how do I" = content ideas with proven demand.
The plays
- Trend radar — daily search runs on your niche keywords, flag videos >100k views in 48h → react fast.
- Competitor teardown — pull a channel's catalog, sort by views/day → their best formats.
- Lead mining — niche channels often list emails in About; combine with contact scrapers.
More tooling: awesome-web-scraping. Agent skills: npx skills add g4dr/apify-skills (repo).
Which niche are you monitoring?
