Website AI agent traffic is not only the agent you deploy. Buyers now send Claude to read pricing pages and book meetings, and this post walks through a small Python checker that finds those requests in an access log, tells the real ones from the fakes, and shows you which of your own rules refused them. By the end you can run it on a day of your log and know whether a real agent was ever turned away, and where.

A drawing on an ivory ground: four page cards of an invented site and a booking calendar, a dotted path along the bottom, a dashed outline of a figure slipping in by the pricing page, and a solid figure stopped by an oxblood bar in front of the calendar

The full story, with a drawn demo of an invented log, is in the write-up on the buyer's agent and the booking page. The code below is trimmed from the real checker.py that ships in the giveaway; nothing here was rewritten for the post.

What Anthropic says about the website AI agent's identity

Anthropic's help article on hosted browsers says three things a parser needs. When Claude browses for a person, the browser keeps its normal User-Agent and adds the token Claude-Agent. The addresses it browses from are published in a SAIL file at https://claude.com/.well-known/sail, under the service Claude-Agent. And the advice to site owners is to match on the token rather than the full string, to verify the source address against the file, and to update rules from the file automatically, because the addresses change. The file follows draft-kelley-webbotauth-sail-00, which is a draft.

Anthropic's crawlers and fetchers (ClaudeBot, Claude-User and Claude-SearchBot) are a separate thing and are managed separately. The checker reports them in their own pile.

Matching a token, not a string

A User-Agent is a list of product names with optional versions. Matching the whole string breaks the first time Chrome ships a new version. Matching the substring Claude-Agent with in is too loose: it would count NotClaude-Agent or Claude-Agentish. So the checker matches a product name with a negative look-behind and look-ahead, case-insensitive:

# A token is a product name: it may be followed by "/version", a space, a ";", ")" or the end. It must # not be glued to more name characters, so "Claude-Agentish" and "NotClaude-Agent" do not count. def _token_re(tok: str) -> re.Pattern: return re.compile(r"(?<![A-Za-z0-9_\-])" + re.escape(tok) + r"(?![A-Za-z0-9_\-])", re.IGNORECASE) AGENT_RE = _token_re(AGENT_TOKEN) CRAWLER_RES = [(t, _token_re(t)) for t in CRAWLER_TOKENS]

The tests pin both directions: the token with a version, at the end or mid-string, in another case, all count; glued names do not. A spaced-out "Claude Agent" is deliberately not the token, and that is printed as a limit rather than guessed at.

Reading the combined log format without dropping a visit

Most servers write the combined format: address, identity, user, time, request, status, bytes, referer, User-Agent. The regex below reads it and tolerates escaped quotes inside the quoted fields:

# combined: host ident user [time] "request" status bytes "referer" "user-agent" LINE_RE = re.compile( r'^(?P<ip>\S+) \S+ \S+ \[(?P<time>[^\]]+)\] "(?P<req>(?:[^"\\]|\\.)*)" (?P<status>\d{3}) (?P<bytes>\S+)' r'(?: "(?P<ref>(?:[^"\\]|\\.)*)" "(?P<ua>(?:[^"\\]|\\.)*)")?' )

The interesting part is what happens when a line does not match. A broken line is the easiest place for a visit to hide, so the checker never drops one silently. A line that does not parse is counted as unreadable, and if the raw line still names a Claude token anywhere it is flagged as unreadable and naming Claude. The same goes for a line that parses but has a stray quote that moves the User-Agent into the wrong field: if the token appears somewhere other than the field that was read, it is counted unreadable, never as other traffic. That rule came out of a hostile review of the first version, which showed that a single extra quote could make a real agent request vanish into "other".

Checking the address against the SAIL file

The SAIL file is TOML, so Python 3.11's tomllib reads it and ipaddress does the membership test. The parser refuses a file with no Claude-Agent service, a service that names a prefix set the file does not define, or a prefix that is not an address. It warns, without refusing, when the schema is not the one it was written for, when the service does not say its list is exhaustive, and when a saved copy is past its own expires_after. The copy fetched on 8 November 2026 listed 9 IPv4 entries and no IPv6.

The membership test itself is short:

def listed(ip: str, sail: dict | None) -> bool | None: """True if listed, False if not, None if the address cannot be read or there is no SAIL file.""" if sail is None: return None if "," in ip: # a forwarded-for list in the address field: refuse to guess which one is the visitor return None try: a = ipaddress.ip_address(ip.strip("[]")) except ValueError: return None if a.version == 6 and a.ipv4_mapped: a = a.ipv4_mapped return any(a.version == n.version and a in n for n in sail["networks"])

Two details matter. An address field that holds a comma-separated list is what you get when a server logs an X-Forwarded-For header, and the client controls that header, so the checker refuses to guess and calls the request unverified. And an IPv4-mapped IPv6 address is unwrapped before the test, so a dual-stack server's log does not turn a real agent into a fake.

Failing closed

If the SAIL file cannot be fetched or parsed, nothing is called real. Every Claude-Agent request goes into an "unverified" pile, the report is still written, and the exit code is 2. That is the only safe direction: a checker that falls back to "token means real" when the list is missing turns every spoofed request into a real one.

Four piles, and blocked keeps its identity

Each Claude request ends up in one pile: real, fake, crawler or blocked. A request is blocked when the status is one of 401, 403, 406, 429, 451 or 503, or a status your rules file names. The first version only counted 403 and 429; the hostile review pointed out that nginx limit_req returns 503 by default and ModSecurity often answers 406, so the list grew.

A blocked request keeps what it was. That is what makes the one number that matters readable straight off the report: how many requests from real agents your site refused, and on which page. The rules file is a plain list (name, words the User-Agent contains, path prefix, status), and the checker names the first rule that explains a refusal. A rule's prefix is a path segment, so /book does not match /bookshelf.

Running it on the invented sample

The giveaway ships an invented 34-line log for a five-page site with one rule on /book that matches the word "agent". The run below uses the saved copy of the address list, so it works offline:

python3 checker.py sample/access.log --rules sample/rules.txt --robots sample/robots.txt --sail sail-snapshot.toml

On that sample it reports 21 Claude-related requests and 13 lines of other traffic. The piles are 5 real, 5 fake, 7 crawler and 4 blocked. The real agent made 8 requests from 2 listed addresses and 3 were refused, all on /book, all by block-scrapers-booking. The fake made 6 requests and 5 were answered. Replaying the rules with listed addresses let through before the scraper rule, 0 real requests are refused and the fake is refused 1 time. The replay is the rules run again over the logged requests, not a recorded outcome.

Screenshot of the demo: the 21 Claude-related requests from the invented log sorted into four piles, real, fake, crawler and blocked, with the booking page ringed where the real agent was refused

Tests that attack it

There are 28 unit tests in test_checker.py, and most of them exist because a review tried to break a claim. Every door in: all three crawler tokens, IPv6 and bracketed addresses, query strings, absolute-form request targets. The error path: statuses other than 403 count as refused, a refusal with no matching rule says so, a status named in the rules file counts. The crash path: an unreadable SAIL file means exit 2 and nothing real, a broken rules line is refused. The same data in another shape: other case, glued names, an escaped quote inside the User-Agent, a stray quote that moves the field, a forwarded-for list. And a full run on the sample that pins every number above.

python3 -m unittest test_checker.py

Using the JSON report

Pass --json report.json and the checker writes everything it found as one document: a summary with the pile counts and the refused requests per page, a per-page breakdown that separates a refused real agent from a refused impostor, and one record per Claude request with its line number, address, page, status, token, pile, the rule that explains a refusal and a plain-language reason. Lines it could not read are listed by number, with a flag when the raw line still names a Claude token. That makes it easy to feed a dashboard, diff two days, or fail a CI job when the count of refused real requests is above zero after a firewall change.

Known limits

Crawler addresses are not verified, because the SAIL file covers the agent only. The checker trusts whatever address your log holds, so behind a CDN or load balancer you need to log the client address. Combined format only. First matching rule wins. The whole log is read into memory, so split a large one by day. And the list format is a draft: if it changes shape, the checker refuses to call anything real and exits 2.

The work split on this build between Claude and Kim, phase by phase, from the build's own work ledger

Get the checker

The script, the invented sample, the tests and a one-page note on what to allow are free from the agent visitor checker page. The write-up with the demo is here, and in Danish at den danske udgave. If you want a website AI agent of your own that buyers and their agents can both reach, that is the website AI agent work I do for B2B companies.