Disclosure: I'm a co-founder of ScrapeUnblocker, one of the APIs measured below. Every number here has a public source you can check.
A lot of websites are still simple to scrape. If you're pulling a blog, a small shop or a public dataset, a normal HTTP request plus a parser is often enough. The work gets very different on the small group of sites that push back: large retailers, travel sites, review platforms and marketplaces. That is where scraping APIs are supposed to justify their cost, and it is also where many of them run into trouble.
Where the market breaks
The most useful public data on this is Proxyway's Web Scraping API Report 2025. They tested 11 scraping APIs against 15 heavily protected websites, around 6,000 URLs per target, at 2 and 10 requests per second.
Averaged across all 11 providers at 2 requests per second, the three hardest targets were:
- Shein: 21.88% success
- G2: 36.63%
- Hyatt: 43.75%
That means the average commercial scraping API got the real Shein page roughly one time in five. These are not hobby projects. They are the established names in the market.
What makes a target hard
It is usually not one single issue. The sites near the bottom of that list tend to stack several defenses together:
- Commercial anti-bot systems. DataDome (G2, leboncoin, Allegro), Kasada (Hyatt), Akamai (Lowe's), Shape (Nordstrom), PerimeterX (Zillow, Walmart), Incapsula (ImmobilienScout24). Each one looks at the TLS handshake, the browser's JavaScript environment and the IP, then decides whether the request looks human.
- In-house protection. Shein, Amazon, Instagram and Google build their own systems, and they change them without warning.
- Blocks that look like success. A challenge page or an "Access denied" body can still come back with HTTP 200. If your check stops at the status code, your success rate is not real.
That last point is important for every number here: success should mean the real content came back, not just a 200.
The established players
You probably know the names already: Bright Data, Oxylabs, Zyte, ScrapingBee, ZenRows, ScraperAPI, Decodo. They are mature products with big teams, and they work well on many sites.
The bigger difference is usually pricing, and on protected targets that can change the real cost quickly:
- Per GB. Oxylabs prices its Web Unblocker by traffic, from $7.20/GB on the entry plan down to $3/GB on enterprise (their pricing page, October 2026). Heavy JavaScript pages cost more.
- Credit multipliers. ScraperAPI's credit table lists Akamai bypass at 10 credits per request (October 2026). The headline price per 1,000 requests applies to easy pages, not to the protected ones you actually came for.
- Subscriptions first. ScrapingBee's entry plan is $49 a month for 250,000 credits, and ZenRows doesn't offer pay-as-you-go on its own site (both October 2026).
None of this is a bad model. It just means the price on the page is often not the price you end up paying on hard targets.
Where a newer API lands
ScrapeUnblocker is new in this market. We built it around a very direct question: how often do you actually get the real page back from the sites that fight hardest? To answer that, we took Proxyway's 2025 target list and ran it against our production API: 6,000 calls per target, 10 at a time, and every response checked for real content, with challenge pages and blocks-as-200 counted as failures. We ran 14 of their 15 targets in July 2026.
| Target | Protection | Field average (Proxyway 2025) | ScrapeUnblocker (July 2026) |
|---|---|---|---|
| shein.com | in-house | 21.88% | 99.3% |
| g2.com | DataDome | 36.63% | 99.9% |
| hyatt.com | Kasada | 43.75% | 99.9% |
| lowes.com | Akamai | 52.57% | 99.4% |
| instagram.com | in-house | 59.54% | 86.7% |
| nordstrom.com | Shape | 61.97% | 86.5% |
| leboncoin.fr | DataDome | 63.83% | 99.6% |
| allegro.pl | DataDome | 66.98% | 95.5% |
| immobilienscout24.de | Incapsula | 71.68% | 99.7% |
| google.com | in-house | 94.78% | 94.4% |
Across all 14 targets we measured 97.0%.
Two caveats are important. First, the field average is the mean of Proxyway's 11 providers, so it shows how difficult a target is rather than rating any single company. Second, this is not a shared test: Proxyway ran at 2 requests per second and we ran at concurrency 10, months apart. Treat the gap as directional. It is still a big gap on the hardest sites, and it is also why I am comfortable showing Instagram and Nordstrom, our two weakest at around 86%, and Google, where we are level with the field.
Don't take my word for it
Every vendor shares its own numbers, including us. The benchmark that matters is the one you run on your own targets:
- Pick the three sites that block you most.
- Send a few hundred real URLs, at the concurrency you'll actually run.
- Count a success only when the content you need is in the response.
Most providers, us included, have a free allowance that is enough for this. An afternoon of testing on your own hardest sites tells you more than any vendor's table, this one included.
Proxyway figures are from Proxyway's Web Scraping API Report 2025; Proxyway did not test ScrapeUnblocker and is not affiliated with it. ScrapeUnblocker figures are our own measurements on our production API in July 2026 (full results: scrapeunblocker.com/benchmark-2026). Competitor details come from each provider's public pages as viewed in October 2026 and may have changed. Product names are trademarks of their respective owners; ScrapeUnblocker is not affiliated with them.
This article was written with AI assistance and checked against the sources above.
