Your website might look beautiful. Fast load times, slick animations, a polished product demo. But if it's built as a JavaScript Single Page Application — React, Vue, Angular, or similar — there's a real risk that ChatGPT, Perplexity, Claude, and the AI tools your buyers use for vendor research receive little or nothing when they try to read it.
For many of these systems, they encounter an empty HTML shell with no readable content — and move on.
This isn't a hypothetical future problem. It's happening right now, today, to thousands of SaaS and FinTech companies that invested heavily in modern web development and didn't realize they'd made themselves significantly less visible to the next generation of search.
How AI Crawlers Actually Work
When Google sends its crawler to your website, it does something remarkable: it actually executes your JavaScript, waits for the page to render, and then reads the result. Google built an entire headless browser infrastructure to do this because JavaScript sites became so prevalent they had no choice.
Many of the systems that power AI-driven answers work differently — though it's worth understanding that "AI search" is not one monolithic thing.
There are at least three distinct layers operating under the hood: training crawlers (like GPTBot, which crawls the web to train OpenAI's models), live browsing fetchers (the mechanism when ChatGPT or Perplexity retrieves a URL in real time), and search-engine crawlers powering AI answer layers (like Bing's crawler, which feeds both traditional results and Copilot). These systems have different architectures, change independently over time, and are not always publicly documented in detail.
What the industry has established through independent testing and crawl analysis: most AI crawlers — particularly training crawlers — use lightweight HTTP fetchers that retrieve your page's raw HTML without executing JavaScript. GPTBot has been independently tested and consistently returns only the initial HTML response with no rendering pipeline. ClaudeBot experiments report the same behavior. PerplexityBot's behavior is less thoroughly documented in primary sources, though available evidence and community testing point in the same direction.
What these crawlers typically receive from a client-rendered React SPA looks like this:
<!DOCTYPE html>
<html>
<head>
<title>My Company | Great Software</title>
</head>
<body>
<div id="root"></div>
<script src="/static/js/main.abc123.js"></script>
</body>
</html>An empty div and a script tag. No product description. No feature list. No pricing. No team page. No blog content. Nothing that can be read, understood, or cited — unless the crawler has access to a cached, archived, or third-party version of your content.
The Traffic Cliff Nobody Diagnosed
Here's where it gets expensive.
A pattern we observe consistently in client audits: companies that migrated from a traditional CMS — WordPress, Webflow, Squarespace — to a modern JavaScript framework, particularly between 2022 and 2024, often saw organic traffic hold relatively steady for months after the migration, then decline sharply. The exact timing varies; some sites hold for six months, others longer, depending on how aggressively the previous content was indexed and how fast AI-powered search behaviors are growing in their buyer segment.
The explanations most teams reached for: "Google's algorithm updated." Or: "We need more backlinks." Or: "Our content isn't good enough."
The more accurate explanation, in many of these cases: Google kept crawling and indexing their content (because Googlebot can execute JavaScript), so Google organic traffic was preserved. But visibility in AI-powered tools, AI-assisted search experiences, and non-Google platforms was quietly eroding — because those systems couldn't read the rendered output.
That second bucket of traffic doesn't show up in Google Search Console, because it was never Google traffic to begin with.
Why This Won't Show Up in Your Google Search Console
Most companies look at Search Console and see their Google organic numbers holding. They conclude things are fine.
They're not looking at the right dashboard.
When someone asks ChatGPT to recommend compliance software, or asks Perplexity which CRM has the best integration ecosystem, or reads a Google AI Overview before clicking a single result — those interactions don't register as Google organic traffic. Some may appear as direct or referral traffic in your analytics. Many leave no trace at all.
This is what makes the JavaScript SPA problem uniquely dangerous for B2B companies: the buyers most likely to use AI for vendor research — technical buyers, VP-level decision-makers at FinTechs, enterprise procurement — are also the buyers most likely to completely bypass the traditional search experience your analytics are optimized to track.
The Fix Is Straightforward — But It's Not Trivial
The good news: this problem has clear, well-established solutions. The bad news: none of them are a 20-minute configuration change.
Server-Side Rendering (SSR) or Static Site Generation (SSG)
This is the most robust long-term solution. Frameworks like Next.js (React), Nuxt (Vue), and SvelteKit render your content on the server before sending it to the browser — meaning every crawler, AI or otherwise, receives fully populated HTML from the first request. If you're building a new site or planning a major rebuild, SSR or SSG should be the default architecture.
Prerendering Service
Tools like Prerender.io detect when a bot hits your site and serve a pre-rendered, cached HTML snapshot instead of the raw SPA shell. This is a faster path for existing sites that can't justify a full architectural overhaul. Done correctly, it's effective. Done sloppily — serving dramatically different content to bots vs. users — it creates cloaking risk, so implementation quality matters.
Static Marketing Layer
A pragmatic approach many companies use: keep the authenticated app experience as a client-rendered SPA, but migrate the public-facing marketing surface — homepage, product pages, pricing, blog, docs — to a static or server-rendered layer. Your SaaS product stays React. Your crawlable content lives in statically generated HTML. Separate deployments, shared domain.
The right choice depends on your existing stack, your dev team's bandwidth, and how much of your site is affected. What's not optional: doing one of the three.
Blocking AI crawlers in robots.txt "for security" without realizing the downstream GEO consequences. Wildcard bot-blocking rules that catch GPTBot and ClaudeBot are more common than you'd think — and they're silent killers for AI visibility.
The Window Is Open — But Not Indefinitely
In a widely cited 2023 forecast, Gartner projected that traditional search engine volume will drop 25% by 2026 due to AI chatbots and virtual agents. It's a projection, not a settled outcome — but the directional signal is already visible in traffic patterns across hundreds of sites we've analyzed.
The companies that address their AI crawlability now are locking in visibility while the pool of competitors is still small. The companies that wait are watching that window close while they debate whether to upgrade their JavaScript framework.
Your buyers are already using AI to research vendors. The question isn't whether AI search matters for your category. The question is whether AI search can find you when it does.
Your Site Might Be Invisible.
Let's Find Out.
We run a free GEO/AIO audit that scores your site across five categories — Technical Crawlability, Content Citation-Readiness, E-E-A-T & Trust, Local GEO Signals, and Content Depth — and delivers a prioritized fix roadmap, not a generic list of "improvements."
Book Your Free SEO Audit →Frequently Asked Questions
Most AI training crawlers, including GPTBot, use lightweight HTTP fetchers that retrieve raw HTML only and do not execute JavaScript. This means client-side rendered SPAs built with React, Vue, or Angular typically appear as empty pages to these crawlers. Live browsing fetchers — used when ChatGPT or Perplexity retrieves a URL in real time — may behave differently, and their architectures are not always fully documented.
Right-click your homepage and choose "View Page Source" (not Inspect). If the body contains only an empty div like <div id="root"></div> with no actual text content, most AI crawlers are seeing little or nothing. You can also disable JavaScript in your browser and reload — if the page goes blank, you have a client-side rendering problem.
GEO stands for Generative Engine Optimization — the practice of optimizing your site to appear in AI-generated answers from tools like ChatGPT, Perplexity, and Bing AI. For JavaScript-heavy sites, the first GEO problem is technical: if AI crawlers can't read your HTML, your content can never be cited, regardless of how good it is.
Not necessarily. A full migration to Next.js, Nuxt, or SvelteKit with SSR or SSG is the most robust long-term solution, but a prerendering service like Prerender.io can solve the problem without a full rebuild. At minimum, migrating your public-facing marketing layer — homepage, product pages, blog — to static or server-rendered HTML will get your key content into AI citation pools.
