Domain availability checking looks simple: type a domain, get a "yes" or "no." But when you need to check hundreds of domains across dozens of TLDs — reliably, quickly, and without paying for a commercial API — the problem gets interesting.

I recently built Site Tools Box, a free online toolbox for webmasters that includes a bulk domain availability checker. Here's how the backend works, and why I chose RDAP over raw WHOIS.

Why Raw WHOIS Doesn't Scale

Traditional WHOIS is a plain-text protocol over port 43. It works, but it has real problems for a modern tool:

  • No standard format. Every registry returns text differently. Parsing "No match for" vs "Not found" vs "Status: available" becomes a fragile regex nightmare.
  • Rate limiting. Some WHOIS servers simply stop responding if you query too fast. There's no standard Retry-After header.
  • No concurrency support. You open a raw TCP socket for each query. That doesn't scale.

For a bulk checker that handles up to 200 domains per request, this approach falls apart quickly.

The Solution: RDAP with WHOIS Fallback

RDAP (Registration Data Access Protocol) is the modern replacement for WHOIS, mandated by ICANN. It's JSON-based, supports HTTP status codes, and most major registries now support it.

The idea is simple: use RDAP first, fall back to WHOIS only when RDAP isn't available.

Step 1: Map TLDs to RDAP Endpoints

Each registry has its own RDAP endpoint. I built a simple lookup table:

function domainRdapEndpoint($tld) { $map = [ 'com' => 'https://rdap.verisign.com/com/v1/domain/', 'io' => 'https://rdap.identitydigital.services/rdap/domain/', 'ai' => 'https://rdap.identitydigital.services/rdap/domain/', 'xyz' => 'https://rdap.centralnic.com/xyz/domain/', // ... and more ]; return $map[$tld] ?? null; }

If the TLD isn't in the map, the tool falls back to a WHOIS socket connection for that domain.

Step 2: Parse the RDAP Response

RDAP returns JSON. A 200 status means the domain is registered, and a 404 means it's available. The JSON body also contains registrar info, registration date, and expiration date:

function parseRdapBody($domain, $body, $source = 'RDAP') { $d = json_decode($body, true); if (!$d) return null; $registrar = ''; if (!empty($d['entities'])) { foreach ($d['entities'] as $entity) { if (!empty($entity['roles']) && in_array('registrar', $entity['roles'])) { foreach ($entity['vcardArray'][1] as $v) { if ($v[0] === 'fn') { $registrar = $v[3]; break; } } } } } return [ 'domain' => $domain, 'registered' => true, 'registrar' => $registrar ?: '-', ]; }

Step 3: Handle Bulk Checks with curl_multi

For bulk requests, I use curl_multi to fire up to 30 concurrent RDAP queries. This is significantly faster than a sequential loop, and it respects the rate limits of most registries:

function domainCheckBatch($domains, $concurrency = 30) { $rdapDomains = []; foreach ($domains as $d) { $tld = substr(strrchr($d, '.'), 1); if (domainRdapEndpoint($tld)) $rdapDomains[] = $d; } for ($i = 0; $i < count($rdapDomains); $i += $concurrency) { $chunk = array_slice($rdapDomains, $i, $concurrency); $mh = curl_multi_init(); // Add handles, execute, collect results } }

Domains without an RDAP endpoint (like .cn or .uk) go through the WHOIS fallback, which uses a raw socket connection on port 43.

Three Things I Learned

1. RDAP is far more reliable than WHOIS. The JSON structure means no more parsing fragile text. HTTP status codes are unambiguous.

2. curl_multi is the right tool for bulk checks. Sequential queries make the tool painfully slow. A concurrency of 30 is the sweet spot before registries start throttling.

3. Always have a fallback. Not every TLD supports RDAP. The WHOIS fallback handles the rest, even if it's slower and messier.

Bonus: Scoring Available Domains

Since I was already building the checker, I added a small scoring algorithm for available domains. It looks at five dimensions:

  • Extension (.com scores highest)
  • Length (shorter is better)
  • Meaning (real English words or pinyin)
  • Pronounceability (vowel-consonant alternation)
  • Character quality (pure letters, no numbers)

It's not a professional valuation — just a rough starting point. But it helps users quickly filter hundreds of available domains.

Try It Out

You can try the domain availability checker here: Site Tools Box

It supports single and bulk checks, and gives available domains a rough value score.

If you're building something similar and have questions about the RDAP implementation, drop a comment below. I'm happy to share more of the code.