Disclosure: SiteGauge, which publishes this article, built the tool used in the second half. The article was drafted by an AI model (Claude). The detection script was tested, and the output and figures shown come from real runs of the tool.
A browser extension will tell you what one website is built with. That stops being useful at about the twentieth tab. If you have a list of 1,000 prospects, competitors or customers and want to know which run Shopify, which use HubSpot, and which still have no cookie banner, you need to do the detection yourself, in bulk.
This article shows how the detection works, gives you a small script you can run today, and then shows the bulk version with real output.
What a website gives away in one request
You do not need a browser, and you do not need to run any JavaScript. A single request for the home page already carries most of the evidence:
| Where to look | Example | What it tells you |
|---|---|---|
| Response headers | x-shopid, server: cloudflare, x-powered-by: Craft CMS | Platform, CDN, server software |
| Cookie names | _shopify_y, CraftSessionId | Platform and tools that set a cookie on the first visit |
| Meta tags | <meta name="generator" content="WordPress 6.8"> | CMS, often with its version |
| Script and stylesheet addresses | /wp-content/…, /_next/static/…, static.klaviyo.com | CMS, framework, every third-party tool that loads a script |
| Inline scripts | the Google Tag Manager snippet | Tag managers and what they are set up to load |
| DNS records | MX, SPF and TXT records of the domain | Email provider, sending services, tools the domain was verified with |
Two of these are easy to get wrong.
Folder names only count in the page's own scripts and stylesheets. A page that links to a PDF in somebody else's /wp-content/ folder is not a WordPress site. Match the folder in <script> and <link> tags, not anywhere in the HTML.
DNS describes the organisation, not the page. A company whose domain is verified with Shopify does not necessarily run this website on Shopify. Keep what you learn from DNS in separate columns.
A script for one website
Here is the idea in 30 lines of Python, standard library only:
import re import urllib.request def detect(url): """Return the technologies a page gives away in its first response.""" req = urllib.request.Request(url, headers={"User-Agent": "tech-check-example/1.0"}) with urllib.request.urlopen(req, timeout=20) as resp: headers = {k.lower(): v for k, v in resp.getheaders()} html = resp.read(2_000_000).decode("utf-8", "replace") found = [] generator = re.search(r'<meta[^>]+name=["\']generator["\'][^>]+content=["\']([^"\']+)', html, re.I) if generator: found.append(f"generator says: {generator.group(1)}") if re.search(r'<(?:script|link)[^>]+/wp-(?:content|includes)/', html): found.append("WordPress") if "x-shopid" in headers or "cdn.shopify.com" in html: found.append("Shopify") if "/_next/static/" in html: found.append("Next.js") if "googletagmanager.com/gtm.js" in html: found.append("Google Tag Manager") if "cloudflare" in headers.get("server", "").lower(): found.append("Cloudflare") return found print(detect("https://wordpress.org/")) This works, and for five technologies on ten websites it is all you need.
It gets harder as the list grows. You will need a few hundred rules instead of five. You will need time-outs, retries on www., and a polite rate per host. Sites that block automated requests have to be told apart from sites that are down. Tools loaded through a tag manager never appear in the HTML, so you have to read the tag manager's container file as well. None of it is difficult. All of it takes time.
The bulk version
We packaged that work as an Apify Actor: Website Technology & Tech Stack Detector. It knows about 500 technologies in more than 60 categories, reads the tag manager container and the domain's DNS, and returns one row per website with the evidence for every match.
The input is a list of domains:
{ "websites": ["apify.com", "crawlee.dev", "wordpress.org"] } With curl
For a short list, one call runs the Actor and returns the rows:
curl -s -X POST \ "https://api.apify.com/v2/acts/sitegauge~website-technology-detector/run-sync-get-dataset-items" \ -H "Authorization: Bearer $APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{"websites": ["apify.com", "crawlee.dev", "wordpress.org"]}' This synchronous endpoint waits at most five minutes. Use it for up to a few hundred websites, and the client below for longer lists.
With Python
import os from apify_client import ApifyClient client = ApifyClient(os.environ["APIFY_TOKEN"]) domains = [line.strip() for line in open("domains.txt") if line.strip()] run = client.actor("sitegauge/website-technology-detector").call(run_input={"websites": domains}) rows = list(client.dataset(run.default_dataset_id).iterate_items()) One thing to watch: this is the syntax of version 3 of apify-client, where call() returns an object. Many examples online still use the older run["defaultDatasetId"], which fails on a current install.
What comes back
This is the row for wordpress.org from a run on 18 November 2026, shortened:
{ "input": "wordpress.org", "finalUrl": "https://wordpress.org/", "status": "analyzed", "technologyCount": 12, "cms": "WordPress", "analytics": "Google Analytics", "tagManagers": "Google Tag Manager", "emailSenders": "Help Scout", "connectedServices": "Google Search Console", "technologies": [ { "name": "WordPress", "category": "CMS", "version": "7.2", "evidence": ["header link: api.w.org", "meta generator: WordPress 7.2", "loads: /wp-content/"] }, { "name": "PHP", "category": "Programming language", "version": null, "evidence": ["implied by WordPress"] }, { "name": "nginx", "category": "Web server", "version": null, "evidence": ["header server: nginx"] } ] } In the same run, apify.com came back with Next.js and React, Google Analytics and Mixpanel, HubSpot, Intercom, OneTrust, Amazon CloudFront, Google Workspace for email and Amazon Route 53 for DNS.
The evidence list is the part that matters most in practice. When a result looks wrong, you can see in one line why the tool said so.
Filtering the list
The category columns (cms, ecommerce, marketing, liveChat, cookieConsent and so on) are plain text, so filtering is one line:
def uses(row, column, name): return name in (row.get(column) or "") shopify_with_klaviyo = [ r["input"] for r in rows if uses(r, "ecommerce", "Shopify") and uses(r, "marketing", "Klaviyo") ] no_cookie_banner = [ r["input"] for r in rows if r["status"] == "analyzed" and not r.get("cookieConsent") ] If you would rather not write code, the Actor's Output tab exports the same table as CSV or Excel.
What it costs and how long it takes
- Price: $2 per 1,000 analyzed websites, plus Apify's small start fee per run. Websites that are blocked, unreachable or disallowed by robots.txt are in the results and are not charged. A free Apify account is enough to try it.
- Speed: about 550 websites per minute at the default memory setting, measured on a mixed list of 736 well-known sites. Slow and dead domains lower that figure, because each one is given up to 20 seconds to answer.
What you cannot see from outside
No tool of this kind can see everything, and it helps to know the limits before you trust a list:
- Only the first request counts. Tools that load after a click, after cookie consent, or only on inner pages such as the checkout are not seen.
- Server-side software that leaves no public trace is invisible. A site can run on almost anything behind a CDN and tell you nothing.
- Some sites refuse automated requests. They show up with the status
blocked, not as a site without technologies. Read thestatuscolumn before you count. - A missing result is not proof of absence. "No analytics found" means none was visible in the first response.
Being a good visitor
For each website the Actor reads robots.txt, requests the home page once and, where the page uses Google Tag Manager, reads the public container file. It identifies itself in its user agent, respects robots.txt and does not try to get around blocks. It reads public pages and public DNS records. It does not collect names, email addresses or other personal data.
If you try it on your own list and a technology is missing or wrongly matched, open an issue on the Actor's Issues tab. Wrong matches are fixed first.