What is answer engine optimization (AEO)?

Search used to end with a list of links. Increasingly it ends with a paragraph. When someone asks ChatGPT which billing platform handles usage-based pricing, or asks Perplexity how to migrate off a legacy CDN, the answer arrives already written — with two or three sources named underneath, and no ranked page of ten blue links anywhere in the interaction.

Answer engine optimization (AEO) is the practice of structuring a website so AI systems can retrieve, parse, and quote its content when composing those answers. It is not a rebrand of SEO. It is a different retrieval surface with different failure modes, sitting on top of the same crawlable web.

Why the retrieval surface changed

A traditional search crawler and an answer engine want different things from your page.

A search crawler indexes your page so it can be ranked and shown as a link. The human clicks through and reads it in your layout, with your navigation, your product callouts, and your conversion path intact.

An answer engine extracts a claim from your page so it can be woven into a sentence somebody else wrote. Your layout is gone. Your navigation is gone. What survives is a fragment of text and, if you are lucky, an attribution link.

That difference produces three concerns SEO never had to think about:

1. Reachability. Answer engines use their own crawlers, with their own user agents, obeying their own interpretation of your robots.txt. Being indexed by Googlebot tells you nothing about whether GPTBot or ClaudeBot can read the same page.
2. Extractability. A page that is beautiful and fast for a human can still extract badly. Content injected client-side, meaning encoded in layout, and claims split across three visual columns all degrade into noise once the DOM is flattened to text.
3. Quotability. Models lift self-contained units. A definition that depends on the previous four paragraphs for context will not be quoted, because quoting it produces a sentence that does not stand up on its own.

The three things AEO actually optimises

Retrieval: can the crawler get the page

Start here, because everything downstream is moot if the fetch fails. In practice the common blockers are a robots.txt that disallows AI user agents (sometimes copied from a template without anyone deciding to), a WAF rule that rate-limits unfamiliar agents, and a bot-management setting that serves a challenge page to anything without a browser fingerprint. All three produce the same symptom — your content is simply absent from AI answers — and none of them show up in a conventional SEO audit.

Parsing: does the content survive extraction

Extraction flattens your page. Whatever meaning lived in visual arrangement is lost, so the meaning has to live in the markup and the prose.

• Semantic headings that describe content rather than section themes: ## Pricing for usage-based billing, not ## Plans.
• One idea per paragraph, with the claim in the first sentence.
• Tables for comparisons that are genuinely tabular, prose for everything else.
• Structured data (Article, FAQPage, Product, Organization) as a machine-readable restatement of what the page already says in prose.
• Content present in the initial HTML response, not assembled after hydration.

Citation: is the claim worth quoting

Quotable content is specific, attributable, and dated. “Faster page loads improve conversion” is not quotable — it is a truism available from a thousand sources. “GPTBot fetches a page and leaves; ChatGPT-User fetches only when a user’s question requires it, which is why the two produce completely different traffic shapes” is quotable, because it is concrete, checkable, and not already in the model’s weights.

Numbers, named entities, version numbers, and dates all raise the odds of being cited. So does taking a position: models composing an answer need something to attribute, and hedged consensus text gives them nothing.

The machine-readable layer

Beyond prose, AEO adds a small set of surfaces that exist purely for machine consumers.

llms.txt is a markdown file at your apex that tells an AI client what your site is, what the important URLs are, and how to think about your product. It is a short, curated map, not a sitemap dump. We cover the format in detail in the llms.txt guide.

Markdown renderings of important pages matter more than people expect. If a client sends Accept: text/markdown and you can hand back clean markdown instead of a 200KB HTML document wrapped around 600 words, extraction is exact rather than best-effort. Every post on this blog is available at /blog/<slug>.md for exactly this reason.

Structured data does the same job in a different dialect: it restates page content in a schema the consumer already knows how to read, which removes the guesswork from extraction.

How you know whether any of it works

This is where most AEO advice stops, and it is the part that matters.

Conventional analytics cannot see AI crawlers. Google Analytics and every other client-side tool run JavaScript in a browser; GPTBot does not execute your analytics bundle, so its visits are invisible. The traffic exists — it is in your server logs, if you keep them long enough and can parse them — but it is absent from the dashboard your team actually looks at.

What you want to measure, in rough order of usefulness:

• Crawl coverage. Which AI operators fetch you at all, and which of your pages they reach. If GPTBot has never requested your pricing page, no amount of copy editing will get that page into an answer.
• Fetch mix. Training crawls versus live retrieval. A page being fetched by ChatGPT-User means a real user’s question is being answered with it right now, which is a fundamentally different signal from a bulk training crawl.
• Demand signals. The paths answer engines re-fetch most often. This is the closest thing AI surfaces have to keyword data, and it tells you which topics to expand.
• Referrals. Sessions arriving from chatgpt.com, perplexity.ai, and similar. Small in volume, and typically higher intent than organic search.

Measuring AI crawler traffic walks through the mechanics of each.

Where to start

If you do nothing else this quarter:

1. Read your robots.txt and make an explicit decision about each AI user agent instead of inheriting a default.
2. Start recording AI crawler hits so you have a baseline before you change anything.
3. Publish an llms.txt that states plainly what your product does and links your five most important pages.
4. Rewrite your top three commercial pages so the first sentence of each section is a standalone, quotable claim.

The first two are diagnostic and the second two are the work. Doing them in that order means you can tell whether the work mattered.