Here's a bug report you'll never get: "ChatGPT says our product doesn't have a free tier."
Nobody files it, because nobody connects the dots. But if your pricing page is a client-rendered React app, there's a decent chance the AI system that answered that question received this:
<!doctype html> <html> <head><title>Pricing | Acme</title></head> <body> <div id="root"></div> <script src="/assets/index-8f3a.js"></script> </body> </html> Then it filled the gap from somewhere else: an old review, a competitor's comparison page, or a guess.
Why this is worse in 2026 than it was in 2020
Google spent years building a JavaScript rendering pipeline. Googlebot runs your JS (eventually), so client-side rendering became "mostly fine for SEO."
AI systems didn't inherit that pipeline. There are now three kinds of AI fetchers hitting your site:
| Type | Examples | Renders JS? |
|---|---|---|
| Training crawlers | GPTBot, ClaudeBot, CCBot | Generally treated as no. Assume raw HTML. |
| AI search crawlers | OAI-SearchBot, Claude-SearchBot, PerplexityBot | Not reliably. Assume raw HTML. |
| Live user fetches | ChatGPT-User, Claude-User, Perplexity-User | Varies by product and changes over time. Don't count on it. |
Vendors don't publish rendering guarantees, and behaviour changes. The only safe assumption is the one that has always been safe: if it's not in the HTML response, it might not exist.
Google's AI features (AI Overviews, AI Mode) use Google's index, which is rendered, so a CSR site can show up there and still be invisible to ChatGPT. That's exactly the kind of gap that's easy to miss.
Test it: raw vs rendered, side by side
This script fetches a URL twice, once as a plain HTTP request (what a non-rendering crawler gets) and once in headless Chromium (what a browser gets), then compares the text.
npm i playwright cheerio npx playwright install chromium // render-gap.mjs import { chromium } from "playwright"; import * as cheerio from "cheerio"; const url = process.argv[2]; const mustContain = process.argv.slice(3); // phrases that MUST be visible to crawlers function textOf(html) { const $ = cheerio.load(html); $("script, style, noscript, template, svg").remove(); return $("body").text().replace(/\s+/g, " ").trim(); } // 1. Raw HTML, as a non-rendering crawler would see it const rawRes = await fetch(url, { headers: { "User-Agent": "Mozilla/5.0 (compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot)" }, }); const raw = await rawRes.text(); // 2. Rendered DOM after JavaScript const browser = await chromium.launch(); const page = await browser.newPage(); await page.goto(url, { waitUntil: "networkidle" }); const rendered = await page.content(); await browser.close(); const rawText = textOf(raw); const renderedText = textOf(rendered); const ratio = rawText.length / Math.max(renderedText.length, 1); console.log(`Status (raw fetch): ${rawRes.status}`); console.log(`Raw text: ${rawText.length.toLocaleString()} chars`); console.log(`Rendered text: ${renderedText.length.toLocaleString()} chars`); console.log(`Raw / rendered: ${(ratio * 100).toFixed(0)}%`); const $raw = cheerio.load(raw); console.log(`<title> in raw: ${$raw("title").text() || "MISSING"}`); console.log(`meta description: ${$raw('meta[name="description"]').attr("content") ? "yes" : "MISSING"}`); console.log(`<h1> in raw: ${$raw("h1").first().text().trim() || "MISSING"}`); console.log(`JSON-LD in raw: ${$raw('script[type="application/ld+json"]').length} block(s)`); for (const phrase of mustContain) { const inRaw = rawText.toLowerCase().includes(phrase.toLowerCase()); console.log(`${inRaw ? "✅" : "❌"} "${phrase}" ${inRaw ? "is" : "is NOT"} in the raw HTML`); } if (ratio < 0.5) { console.log("\n⚠️ Less than half of the visible text exists without JavaScript."); process.exitCode = 1; } node render-gap.mjs https://your-site.com/pricing "Free plan" "per seat" "SOC 2" Status (raw fetch): 200 Raw text: 142 chars Rendered text: 6,418 chars Raw / rendered: 2% <title> in raw: Pricing | Acme meta description: MISSING <h1> in raw: MISSING JSON-LD in raw: 0 block(s) ❌ "Free plan" is NOT in the raw HTML ❌ "per seat" is NOT in the raw HTML ❌ "SOC 2" is NOT in the raw HTML ⚠️ Less than half of the visible text exists without JavaScript. The mustContain phrases are the important part. Pick the facts you'd be upset to see an AI get wrong (price, free tier, integrations, compliance) and make them a test.
Also check the status code. If the raw fetch with an AI user agent returns 403 or a challenge page while your browser gets 200, your problem is the CDN or WAF, not rendering. Look at your bot-management rules before changing any code.
The fixes, by framework
Next.js (App Router)
Server Components render on the server by default, so most App Router pages are already fine. The usual culprits:
// ❌ Whole page opted into client rendering, data fetched in useEffect "use client"; export default function Pricing() { const [plans, setPlans] = useState([]); useEffect(() => { fetch("/api/plans").then(r => r.json()).then(setPlans); }, []); return <PlanTable plans={plans} />; } // ✅ Server Component fetches data; only the interactive toggle is a client component import { BillingToggle } from "./billing-toggle"; // "use client" inside export const revalidate = 3600; // static, refreshed hourly export default async function Pricing() { const plans = await getPlans(); return ( <main> <h1>Pricing</h1> <BillingToggle /> <PlanTable plans={plans} /> </main> ); } Also watch for dynamic(() => import(...), { ssr: false }) around content (it's fine for widgets, not for text), and for content that only appears inside tabs or accordions that mount on click. Render it all and hide it with CSS instead.
Vite + React SPA
You have three options, from least to most work:
- Prerender the routes that matter at build time (marketing pages, docs, pricing), for example with
vite-plugin-prerenderor a small Playwright script that saves rendered HTML for a list of routes intodist/. - Move the marketing site to a static-first framework (Astro, Next.js static export) and keep the SPA for the logged-in app at
app.example.com. - Adopt SSR (React Router framework mode, TanStack Start) if most of the product is public content.
For most teams, option 2 is the right answer: the app doesn't need to be crawlable, and the marketing site does.
Astro, Eleventy, Hugo
Static by default. Just be careful with client-only islands (client:only) that contain content rather than interactivity.
What about dynamic rendering?
Serving prerendered HTML only to bots (the old "dynamic rendering" pattern) still works technically, but Google describes it as a workaround, not a recommended solution. It also creates two versions of every page that can drift apart, and serving different content to bots is the definition of cloaking. Prefer SSR or static generation, which give everyone the same HTML.
Put the check in CI
Run the script against a preview deployment on every PR that touches marketing or docs routes:
# .github/workflows/render-gap.yml name: Crawlability on: pull_request jobs: render-gap: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: { node-version: 22 } - run: npm ci && npx playwright install --with-deps chromium # PREVIEW_URL comes from your deploy provider's preview step - run: node scripts/render-gap.mjs "$PREVIEW_URL/pricing" "Free plan" "per seat" - run: node scripts/render-gap.mjs "$PREVIEW_URL/docs/quickstart" "npm install" That one check would have caught every "the AI said we don't have X" problem I've seen.
Beyond rendering
Server-side rendering gets your content into the HTML. Whether AI crawlers can reach it at all also depends on robots.txt, your CDN's bot rules, rate limits and status codes, and whether the page is indexed in Google and Bing. Vista AI's page audit checks those things together with the raw-vs-rendered gap for a URL. If you'd rather script it yourself, the render-gap.mjs above plus a robots.txt check covers the most common failures.
Run the script on your pricing page and share the raw/rendered percentage in the comments. Bonus points if it's under 10%.
Next in the series: How AI assistants decide which dev tool to recommend.


