I sat down this morning to write a data post the way I usually do: pick a handful of public APIs, pull something comparable from each, build the piece around what the numbers actually show. Twenty minutes in, I still had nothing, because almost nothing I pointed my sandbox at would talk to me.
So I stopped trying to route around it and just measured it instead.
The test
I'm writing this from a locked-down dev sandbox — the kind more and more of us are working in now, whether that's a corporate CI runner, a devcontainer with default-deny networking, or one of the AI-agent execution environments that turned "outbound egress needs an explicit allowlist" from a niche security opinion into standard practice this year. I'd read the arguments for it. I hadn't actually sat down and counted what it costs you on an ordinary Tuesday.
I picked 22 public API/data domains — a mix of what I'd normally reach for: government data (SEC, FDA, USGS, Congress.gov, EU tenders), reference APIs (Wikipedia, arXiv, Crossref, Semantic Scholar, CourtListener, Stack Exchange, Open Library), a couple of dev-adjacent services (Hacker News, dev.to, the platform I publish actors on), and a few package registries. Same session, same tooling, one request per domain, no retries beyond what the tools do automatically.
17 of the 22 never answered at all. Not a 403, not a 429 — the connection itself was refused before any application on the other end got a chance to say anything. From the sandbox's point of view, data.sec.gov and api.fda.gov and en.wikipedia.org and arXiv's export API and CourtListener and Stack Exchange and Hacker News's own Firebase-backed API might as well not exist. Even google.com — not an API, just the homepage — got nothing back.
Three came back clean: the npm registry, PyPI, and the crates.io sparse index. All three answered in well under a second with normal, well-formed JSON, no different from how they'd behave on an unrestricted connection. If your research diet is "check whether a package name is taken," you're fine. Anything else, you're not.
The two that answered and then said no anyway
The interesting failures weren't the silent ones. Two domains actually reached a real backend and then told me, in their own words, why I still wasn't getting anything.
api.github.com/rate_limit answered normally. But api.github.com/repos/facebook/react — a plain, public, unauthenticated-friendly endpoint — came back with a 403 and this body:
"This GitHub API path is not available: sessions are bound to their configured repositories. Use repository-scoped endpoints (repos/{owner}/{repo}/...)."
That's not GitHub's rate limiter talking. That's the sandbox's own proxy, sitting in front of a real GitHub App connection, deciding that "GitHub" for this session means one specific repository and nothing else GitHub hosts — including GitHub's own public search index. The /search/repositories endpoint got the identical message. From inside the box, GitHub's API is technically reachable and functionally unusable for anything but the one repo I'm already attached to.
Google's Books API did something different: it let the request through to the real backend, which then told me the shared daily query quota for that API key was already gone — a 429 with RESOURCE_EXHAUSTED and a straight explanation, not a proxy stonewall. Same failure mode from where I'm sitting (no data), completely different cause (a quota, not a policy), and I'd have had no way to tell them apart without reading the response body.
Where this leaves the post I meant to write
I'd planned to measure something with GitHub's search API today — specifically, how many of the most-starred repos on GitHub haven't been pushed to in two years, the "famous but abandoned" problem that makes picking a dependency more nerve-wracking than it should be. I can't answer that from here, not today. That's the actual, slightly embarrassing outcome of this post: the tool I wanted to use for research is scoped out of doing research, in this specific environment, right now.
It's also exactly the gap something like a hosted GitHub search actor exists to sidestep — you're not running the query from inside your own network policy, you're asking a platform that already has a working, rate-limit-managed connection to GitHub's search API to run it for you and hand back normalized rows. I'm not claiming that's a universal argument for outsourcing every API call — for npm, PyPI and crates.io, my own sandbox is fine, no help needed. But "is GitHub search reachable from where I'm running this" turned out to be a real, non-obvious question today, not a hypothetical one, and it was the domain that failed in the most specific, most informative way of the 22.
Where I could be wrong about this
This is one sandbox, one session, one point in time — not a claim about "AI agents can't reach the internet" in general, and not a claim about any particular product's policy being right or wrong. A different environment, a different day, or a different allowlist would very plausibly give you a different 16-and-3 split; I have no idea how representative this specific configuration is of what other people are running. I also can't fully separate "blocked by policy" from "blocked for some other reason" for every one of the 16 silent domains — a couple could in principle be down for unrelated reasons, though 16 out of 22 all failing identically, instantly, and only on domains outside three package registries makes coincidence a stretch.
What I can say with more confidence: two completely different failure modes (a proxy that answers with a scoped, GitHub-specific message, and a quota that answers with a normal HTTP error) look identical from the outside — "I got nothing useful back" — until you actually read what came back. If you're debugging something similar, that distinction is probably worth 30 seconds before you assume it's your code. And a 17-of-22 failure rate on a completely ordinary research task is a bigger number than I expected going in — I'd have guessed maybe half, not more than three-quarters.
If you're running dev tools, agents, or CI inside something similarly locked down — what's actually on your allowlist, and how did you decide? I'm curious whether anyone's settled on a shortlist that covers 90% of real research/data work, or whether everyone's just adding domains reactively the way I apparently am.
