If you give a coding or research agent a YouTube or TikTok URL today, the naive pipeline looks straightforward: run yt-dlp, download the MP4, and pass the video bytes to a vision-capable model.
In production, that pipeline breaks for four separate reasons before your agent even reads a single sentence:
- Most frontier coding agents cannot take raw video files at all. Anthropic Claude accepts images (up to 600 per request within a 128 MB cap) but has no native video or audio input. OpenAI's Responses API similarly takes images and documents rather than arbitrary social video URLs. Even when a model family accepts video natively, you still need a deterministic, cacheable text-and-frame layer if your pipeline feeds Claude, GPT, DeepSeek, or a RAG vector store.
- Token and bandwidth costs scale linearly with video length. On a 30-minute video at high visual resolution, dense frame sampling burns hundreds of thousands of tokens when 90% of the answer sits in the spoken transcript and three specific slides.
- Platform CDN links are IP-bound and short-lived. Passing a raw
googlevideo.comortiktokcdn.comURL straight to an external LLM API returns403 Forbiddenbecause the media signature is tied to the IP address that negotiated the player session. - Datacenter IPs hit silent bot traps. If you deploy an extraction service onto cloud infrastructure such as Hetzner, Vultr, or AWS, YouTube responds to most real videos with
LOGIN_REQUIREDwhile letting famous test fixtures succeed.
We built VidCrawl around progressive video disclosure: keep Layer 1 purely deterministic (zero LLM tokens), send lightweight storyboard grids alongside timestamped text, and fetch high-resolution pixels only at the exact timestamps where text alone leaves ambiguity.
Here is how the extraction engine works under the hood, including the production failure modes we hit along the way.
Trap #1: The Hetzner LOGIN_REQUIRED and Rickroll (dQw4w9WgXcQ) Whitelist Trap
Early in testing from our Hetzner (AS24940) server, our YouTube smoke tests passed against dQw4w9WgXcQ, and then failed immediately when real users pasted real videos like 1E3tv_3D95g, UF8uR6Z6KLc, or aircAruvnKk.
From flagged cloud ASNs, YouTube's /youtubei/v1/player and watch HTML return:
playabilityStatus.status = "LOGIN_REQUIRED" reason = "Sign in to confirm you're not a bot" Meanwhile, dQw4w9WgXcQ continues returning OK from the exact same datacenter IP. If your CI or /healthz check tests dQw4w9WgXcQ, your monitoring stays green while 90%+ of real production URLs fail.
A second mistake is masking that failure: returning ok: true with zero transcript segments when playabilityStatus.status === "LOGIN_REQUIRED". VidCrawl excludes dQw4w9WgXcQ from live reliability suites (testing non-whitelisted URLs such as 1E3tv_3D95g instead) and raises a typed YOUTUBE_BOT_CHALLENGED (502) error whenever /player returns LOGIN_REQUIRED rather than returning an empty success payload.
Fix #1: Split-Routing /next, Sticky Proxy /player, Direct json3, and i.ytimg.com
Routing an entire video download through a residential proxy destroys unit economics. Instead, VidCrawl splits every YouTube scrape across four distinct network paths in packages/engine/src/youtube.ts:
POST /v1/scrape (1 credit, 0 LLM tokens) | +---> 1. Direct server IP: POST /youtubei/v1/next (WEB client) | -> visitorData, attributedDescription, outgoing links, macro-marker chapters | -> 0 proxy bytes (skips ~1.2 MB watch HTML) | +---> 2. Sticky residential proxy: POST /youtubei/v1/player | -> Cycles VISIONOS / ANDROID_VR / IOS clients on deterministic sticky port | -> ~20-40 KB compressed JSON per call | +---> 3. Direct server IP: GET i.ytimg.com/sb/... (highest available L3/L2 storyboards) | -> Sprite sheets of 3x3 or 5x5 tile grids (0 proxy bytes) | +---> 4. Direct server IP: GET timedtext?fmt=json3 -> Clean tStartMs + dDurationMs segments (0 proxy bytes) Three implementation details matter here:
- Deterministic sticky proxy ports (
getStickyProxyUrl): For DataImpulse (gw.dataimpulse.com:823), ports10000..19999map to sticky residential sessions. Hashing${videoId}:${attempt}onto a sticky port keeps the/playernegotiation and any subsequent IP-bound media or audio fetch on the same exit IP while rotating across videos and retry attempts. -
json3captions instead of naive WebVTT: YouTube's auto-generated.vtttracks use a two-line rolling display window where every cue repeats the previous line, causing duplicated sentences in downstream transcripts. Fetching&fmt=json3(withsrv3and XML fallbacks) yields discreteevents[].segs[].utf8entries keyed bytStartMsanddDurationMs. - Bun runtime requirement for
fetch({ proxy }): Bun natively supportsfetch(url, { proxy }), whereas Node ignoresproxyonRequestInitand falls back to direct egress. VidCrawl checks for Bun at runtime whenVIDCRAWL_PROXY_URLis configured so traffic never leaks directly by mistake.
On our recorded 18:22 YouTube sample (WWDC 2025 Impressions: Liquid Glass!, 1E3tv_3D95g), Layer 1 produces 26 KB of Markdown (465 transcript segments and 9 chapters) plus 237 KB across 5 storyboard sheets (112 tiles sampled every 10 seconds). An agent can read the entire talk and scan every visual transition before requesting a single full-resolution frame. If you want to test the Markdown output directly on a browser URL, see our Video to Markdown tool.
Fix #2: ISO-BMFF sidx Byte-Range Frame Slicing (POST /v1/frames)
When a storyboard grid shows an unreadable 14px IDE buffer, chart axis, or settings screen, the agent calls POST /v1/frames with timestamps_sec (1 to 20 timestamps, max_width from 320px to 1920px).
To avoid downloading a multi-hundred-megabyte MP4 just to grab two screenshots, packages/engine/src/youtube.ts caches the best MP4 DASH stream from /player for 15 minutes, preferring 720p streams (compareFrameStreams scores by short side so vertical 720x1280 Shorts do not stall on giant 1080x1920 segments) along with initRange and indexRange.
Then packages/engine/src/frames.ts performs exact byte-range slicing:
- Fetch the container header (
0..indexRange.end) capped atMAX_HEADER_BYTES = 2 MB. - Parse the ISO-BMFF
sidx(Segment Index Box,0x73696478) in TypeScript (findSegmentByteRangeFromSidx), validating version 0 and version 1 reference entries and accumulating timestamps acrosstimescaleticks untiltargetSeclands inside[startSec, endSec). - Fetch only
initRangeplus that single 5-second GOP fragment[byteStart, byteEnd]capped atMAX_MEDIA_BYTES = 16 MB, enforcing manual redirects (redirect: "manual") on every hop so redirects can never leavegooglevideo.com, and reusing the exactstickyProxyUrlfrom/player. - Decode in memory with
ffmpegandsharp: Concatenate[initBuf, segBuf], pipe intoffmpeg(select=gte(n,${targetFrameIdx}) -frames:v 1), resize tomax_width, and encode as JPEG (quality: 88).
If the stream header or sidx segment is unavailable, /v1/frames falls back explicitly to storyboard_crop by extracting the closest tile from the storyboard JPEG via sharp and labeling the source field as "storyboard_crop" rather than "ffmpeg_seek".
Handling Short-Form Video (TikTok, Instagram Reels, X)
Short-form clips do not expose YouTube-style sidx byte ranges or static i.ytimg.com storyboards, and they frequently deliver critical text as burned-in overlays rather than spoken words.
For TikTok, VidCrawl uses a two-rung pipeline (packages/engine/src/tiktok.ts and short-form-media.ts) that powers both /v1/scrape and our public TikTok transcript tool:
- Rung 1 (Zero-
yt-dlpHydration JSON): Fetch the page HTML and parse<script id="__UNIVERSAL_DATA_FOR_REHYDRATION__">(webapp.video-detail.itemInfo.itemStruct), pulling native ASR WebVTT tracks directly fromsubtitleInfoswhen present. - Rung 2 (Lowest-Bitrate H.264 + Direct-First Fallback): When building the
3x3contact sheet (960x1704,320x568tiles viaffmpeg) or running Whisper ASR, select lowest-bitrateh264/avcstreams rather thanbytevc1(which on TikTok frequently omits audio). Downloads run directly from the server IP first and fall back to the residential proxy only when the CDN answers HTTP 403 (~17%of TikTok clips from production egress).
Instagram Reels (embed/captioned/ HTML + bounded clip processing) and X posts (cdn.syndication.twimg.com/tweet-result + video.twimg.com clips) follow the same bounded 50 MB / 600s pipeline and return the exact same JSON schema.
Measured Reliability on Hetzner Egress (300-URL Benchmark)
Instead of guessing reliability, we run a deterministic 300-URL evaluation suite (eval/reliability/run.ts, 60 public URLs per platform set, concurrency 3, 16.9 minutes total) from Hetzner egress (eval/reliability/results/2026-11-16-hetzner-final.md):
| Platform Set | Scrape OK | Timed Transcript | Any Text | Storyboard | Frame OK (1280px) | Scrape p50 / p95 | Proxy KB / Scrape |
|---|---|---|---|---|---|---|---|
| YouTube | 97% (58/60) | 97% (56/58) | 97% (56/58) | 100% (58/58) | 95% (19/20) | 2.4s / 10.4s | 189 KB |
| YouTube Shorts | 98% (59/60) | 92% (54/59) | 92% (54/59) | 100% (59/59) | 84% (16/19) | 3.1s / 14.7s | 55 KB |
| TikTok | 100% (60/60) | 95% (57/60) | 95% (57/60) | 88% (53/60) | 85% (17/20) | 7.7s / 24.1s | 229 KB |
| Instagram Reels | 100% (60/60) | 93% (56/60) | 93% (56/60) | 100% (60/60) | 100% (20/20) | 11.5s / 27.6s | 0 KB |
| X Video | 100% (60/60) | 82% (49/60) | 100% (60/60) | 100% (60/60) | 100% (20/20) | 2.7s / 8.7s | 0 KB |
Observed failure modes in that run were explicit and untruncated: 2 VIDEO_UNAVAILABLE and 1 fell_back_to_storyboard_crop on YouTube, 1 scrape UPSTREAM_TIMEOUT and 3 frame UPSTREAM_TIMEOUT on Shorts, and 3 fell_back_to_storyboard_crop on TikTok. Every failed request before model completion is automatically excluded from billing.
Connecting via Remote HTTP MCP or npx -y @vidcrawl/mcp
The VidCrawl MCP server exposes five tools that mirror the progressive-disclosure workflow:
-
vidcrawl_scrape(1 credit): Metadata, chapters, transcript, storyboard sheets, and Markdown. -
vidcrawl_frames(1 credit): Up to 20 sharp JPEG frames at exacttimestamps_sec. -
vidcrawl_search(1 credit, +1 per scraped result): YouTube discovery with optional Layer 1 scrape enrichment. -
vidcrawl_agent(3 / 5 / 10 credits): Bounded visual + spoken synthesis across 1 to 10 URLs with[MM:SS]citations and optional JSON schema. -
vidcrawl_account(0 credits): Check remaining credits and endpoint prices.
Option A: Remote Streamable HTTP (Zero Local Dependencies)
In Claude Code:
claude mcp add --transport http vidcrawl https://vidcrawl.com/mcp \ --header "Authorization: Bearer $VIDCRAWL_API_KEY" In OpenAI Codex:
codex mcp add vidcrawl --url https://vidcrawl.com/mcp # In ~/.codex/config.toml under [mcp_servers.vidcrawl], add: # bearer_token_env_var = "VIDCRAWL_API_KEY" In Cursor or Windsurf (.cursor/mcp.json):
{ "mcpServers": { "vidcrawl": { "url": "https://vidcrawl.com/mcp", "headers": { "Authorization": "Bearer YOUR_VIDCRAWL_API_KEY" } } } } Option B: Local Stdio MCP and CLI (@vidcrawl/mcp)
# Run as a stdio MCP server VIDCRAWL_API_KEY="$VIDCRAWL_API_KEY" VIDCRAWL_API_URL="https://vidcrawl.com" npx -y @vidcrawl/mcp mcp # Or call the CLI directly from a shell script npx -y @vidcrawl/mcp scrape 'https://www.youtube.com/watch?v=1E3tv_3D95g' npx -y @vidcrawl/mcp frames 'https://www.youtube.com/watch?v=1E3tv_3D95g' --at 150,365 Pricing and Alternatives
New accounts include 100 free credits with no card required, and one-time credit packs start at $9 / 1,000 credits and $29 / 4,000 credits (credits never expire). Every /v1/scrape, /v1/frames, and /v1/search call costs 1 credit regardless of whether a clip is 15 seconds or 2 hours long, and /v1/agent costs 3, 5, or 10 credits depending on effort rather than per-minute billing. If you are evaluating alternatives on per-minute vs per-call economics, SDKs, and platform coverage, check our side-by-side breakdown on VidCrawl vs Supadata.
Try the live playground or create an API key at VidCrawl.
