This article was generated with AI. The examples use a synthetic example.com site so you can adapt them to a site you operate.
A site can return HTTP 200 for a sitemap URL while serving the Blog homepage or an outdated XML file. If your deployment check stops at the status code, both can pass. After moving a Blog to a static site, compare the final address and the XML with the pages the release is supposed to publish.
Record the expected result before the cutover
Write down the sitemap address you expect clients to request, the final XML address and the article URLs that must appear. Get the expected article URLs from the release inventory so that the check can catch a valid but stale sitemap. If you retain an older sitemap address as a redirect, check it separately against the same XML destination.
| Request | Expected response |
|---|---|
https://example.com/blog/sitemap.xml | HTTP 200 at the same address with the expected article locations |
https://example.com/old-blog-sitemap.xml | Ends at https://example.com/blog/sitemap.xml with the same expected locations |
An XML sitemap for pages uses a namespaced urlset, with a loc inside each url. An index uses sitemapindex and points to child sitemaps. Run the check below against a child sitemap that lists the article URLs. Sitemaps XML protocol
Verify the XML and its contents
Save this as sitemap_check.py. It uses Python’s standard library to follow redirects and check the final address. Pass a few article URLs to check that they appear, or pass --manifest to compare the full inventory. Both modes reject duplicate sitemap entries.
import argparse import xml.etree.ElementTree as ET from pathlib import Path from urllib.request import urlopen NS = "{http://www.sitemaps.org/schemas/sitemap/0.9}" MAX_BYTES = 50 * 1024 * 1024 def check_xml(body, expected_urls, *, exact=False): if len(body) > MAX_BYTES: raise ValueError("Sitemap is larger than 50 MiB") root = ET.fromstring(body) if root.tag != NS + "urlset": raise ValueError(f"Expected a sitemap urlset, got {root.tag!r}") listed_urls = set() for entry in root.findall(NS + "url"): loc = entry.findtext(NS + "loc", default="").strip() if not loc: raise ValueError("A URL entry has no location") if loc in listed_urls: raise ValueError(f"Duplicate URL: {loc}") listed_urls.add(loc) expected = set(expected_urls) missing = sorted(expected - listed_urls) unexpected = sorted(listed_urls - expected) if exact else [] if missing or unexpected: details = [] if missing: details.append(f"{len(missing)} missing, including {missing[0]}") if unexpected: details.append(f"{len(unexpected)} unexpected, including {unexpected[0]}") raise ValueError("URL inventory mismatch: " + ". ".join(details)) return len(listed_urls) def main(): parser = argparse.ArgumentParser() parser.add_argument("request_url") parser.add_argument("final_url") parser.add_argument("page_urls", nargs="*") parser.add_argument("--manifest", type=Path, help="Exact URL inventory, one URL per line") args = parser.parse_args() if bool(args.page_urls) == bool(args.manifest): parser.error("Pass page URLs or --manifest") try: expected_urls = args.page_urls if args.manifest: expected_urls = [ line.strip() for line in args.manifest.read_text(encoding="utf-8").splitlines() if line.strip() ] if not expected_urls or len(expected_urls) != len(set(expected_urls)): raise ValueError("The manifest needs a nonempty list of unique URLs") with urlopen(args.request_url, timeout=15) as response: if response.status != 200 or response.url != args.final_url: raise ValueError(f"Unexpected response: {response.status} {response.url}") count = check_xml(response.read(MAX_BYTES + 1), expected_urls, exact=args.manifest is not None) except (OSError, ValueError, ET.ParseError) as error: raise SystemExit(f"FAIL {args.request_url}: {error}") from None print(f"PASS {args.final_url}: {count} unique URL entries") if __name__ == "__main__": main() Run the first example to check a single article. For a full cutover check, save the expected canonical URLs from the release inventory in a UTF-8 file with one URL per line. The second example checks the old sitemap address, the final address and the exact URL list. URL order can differ.
python3 sitemap_check.py \ https://example.com/blog/sitemap.xml \ https://example.com/blog/sitemap.xml \ https://example.com/blog/first-article/ cat > expected-blog-urls.txt <<'EOF' https://example.com/blog/first-article/ https://example.com/blog/second-article/ EOF python3 sitemap_check.py \ https://example.com/old-blog-sitemap.xml \ https://example.com/blog/sitemap.xml \ --manifest expected-blog-urls.txt Make sure the check fails
Point it at a known HTML route and use that route as the expected destination. The final-address check passes, then XML parsing or the root-element check rejects the payload. Repeat with a valid sitemap and an expected article URL that it lacks. That catches the case where a cache or an old publishing job serves well-formed XML from the wrong release.
Add a retired article URL to a test sitemap and leave it out of the release manifest. The exact check reports that URL as unexpected. If the test sitemap also drops a required article, the check reports both problems in the same run.
Check the first redirect
If the old sitemap should redirect directly to its replacement, check the first response before checking the XML. This example requires a permanent redirect and an exact destination, including both occurrences of the query parameter.
set -eu legacy='https://example.com/old-blog-sitemap.xml?ref=1&ref=2' expected='https://example.com/blog/sitemap.xml?ref=1&ref=2' response=$(curl --disable --silent --show-error --max-time 15 \ --output /dev/null --write-out '%{http_code} %{redirect_url}' "$legacy") case "$response" in "301 $expected"|"308 $expected") printf '%s\n' "$response" ;; *) printf 'Unexpected first response %s\n' "$response" >&2; exit 1 ;; esac The check accepts HTTP 301 or 308. redirect_url resolves a relative Location so the comparison uses the full address. Run the Python check next to verify the replacement’s XML. For a migration that intentionally retains an intermediate hop, record the expected status and destination at each hop. curl reference · Google’s redirect guidance
Verify the canonical on each article
The sitemap can list https://example.com/blog/first-article/ while the article template points its canonical at https://example.com/. When both URLs return HTTP 200, test the article’s HTML head against the address in the release inventory. A shared template can repeat the same error across the whole Blog. Google’s canonical guidance
Save this as article_canonical.py. It checks the final response and requires one matching absolute canonical in the original HTML head. Use it on HTML templates that emit a canonical in the server response.
import sys from html.parser import HTMLParser from urllib.request import urlopen class CanonicalHead(HTMLParser): def __init__(self): super().__init__() self.in_head = False self.template_depth = 0 self.links = [] def handle_starttag(self, tag, attrs): if tag == "head": self.in_head = True elif tag == "body": self.in_head = False elif self.in_head and tag == "template": self.template_depth += 1 elif self.in_head and not self.template_depth and tag == "link": fields = dict(attrs) relation = (fields.get("rel") or "").lower().split() if "canonical" in relation and not {"hreflang", "lang", "media", "type"} & fields.keys(): self.links.append(fields.get("href")) def handle_endtag(self, tag): if tag == "head": self.in_head = False elif tag == "template" and self.template_depth: self.template_depth -= 1 def check_head(html, expected): parser = CanonicalHead() parser.feed(html) parser.close() if parser.links != [expected]: raise ValueError(f"{expected}: expected one matching canonical, found {parser.links!r}") def check_article(expected): limit = 2 * 1024 * 1024 with urlopen(expected, timeout=15) as response: if response.status != 200 or response.geturl() != expected: raise ValueError(f"{expected}: got {response.status} at {response.geturl()}") if response.headers.get_content_type() != "text/html": raise ValueError(f"{expected}: expected HTML") body = response.read(limit + 1) if len(body) > limit: raise ValueError(f"{expected}: HTML exceeds the 2 MiB check limit") check_head(body.decode(response.headers.get_content_charset() or "utf-8"), expected) print(f"OK {expected}") if __name__ == "__main__": if len(sys.argv) != 2: raise SystemExit("Usage: python3 article_canonical.py HTTPS_ARTICLE_URL") try: check_article(sys.argv[1]) except (OSError, ValueError) as error: raise SystemExit(str(error)) from error python3 article_canonical.py https://example.com/blog/first-article/ Try the check against a test article whose canonical points to the homepage, then fix the template and run it again. If your site also sets an HTTP Link canonical or adds one with JavaScript, compare those signals separately with the same expected address.
Verify search discovery separately
After the deployment check passes, use Search Console to confirm Google processed the advertised sitemap. Inspect a few important article URLs separately to see whether Google has crawled or indexed them. Record the XML inventory, the article canonicals, sitemap processing and URL Inspection separately in your release notes. Google’s sitemap guidance
