Field notes

Your site looks fine in a browser and empty to a crawler

Content assembled by JavaScript after load is invisible to anything that reads the served HTML. How to tell whether that applies to you, and what to do about it.

There is a specific failure that produces excellent-looking pages and terrible machine readability, and the people responsible for the site almost never see it. The page renders perfectly in every browser they test. The content is simply not in the document the server sent.

What actually arrives

When a client-rendered application is requested, the server often returns a nearly empty shell — a div for the app to mount into, and script tags. The browser downloads the scripts, executes them, fetches data, and builds the page. By the time a human looks, everything is there.

An automated reader may do none of that. Executing JavaScript is expensive at scale, and many crawlers either skip it, defer it to a later pass that may not come, or apply a time budget that a slow application exceeds. What they see is the shell.

How to check in thirty seconds

Do not use your browser's inspector — it shows the page after scripts have run, which is exactly the view that hides the problem. Two reliable methods:

  1. View source rather than inspect. In most browsers, Ctrl+U or Cmd+Option+U shows the document as delivered. Search it for a distinctive sentence from your page. If it is not there, it was added by script.
  2. Fetch the page from a terminal with curl and read what comes back. No JavaScript runs, so what you see is what a non-executing reader sees.

What to do about it

The fix is to send meaningful HTML in the first response. In order of preference:

Server-side rendering or static generation

Every major framework supports rendering on the server or generating pages at build time. For content that does not change per visitor — articles, product descriptions, documentation — static generation is both the fastest option for readers and the most robust for machines. This is the real fix.

Partial rendering of what matters

A full migration is often unrealistic. It is usually possible to server-render the parts that carry meaning — headings, body copy, structured data, key facts — while leaving genuinely interactive components to the client. A dashboard widget does not need to be in the HTML. Your product description does.

What not to do

Do not serve different content to crawlers than to people. Beyond being against the guidelines of essentially every platform, it is fragile, and detection of the practice tends to be punitive rather than corrective.

The structured data trap

One consequence deserves separate mention because it is so commonly missed. Structured data injected by a tag manager or an analytics-style script is subject to exactly the same limitation. Teams frequently add Organization or Article markup through a tag manager, verify it in a testing tool that executes JavaScript, and conclude it is working.

It may be working for readers that execute scripts and absent for everything else. If structured data matters — and identity markup does — put it in the served HTML.

A note on proportion

This is not an argument against modern frameworks. It is an argument for knowing which of your pages carry meaning, and confirming that the meaning is present before scripts run. For most sites that is a handful of page types, and checking them takes minutes.

Sources

Check your own site

Run the audit described in these notes against your own website. No signup, evidence included for every check.

Run a free audit