Agent Analyzer

Retrievability comes before every other factor. A page an answer engine cannot fetch cannot be read, summarised or cited. Guidance

A noindex directive in a meta tag or an X-Robots-Tag header removes the page from search indexes, and the AI search products built on them follow it. Guidance

Most AI crawlers do not execute JavaScript. Content that only appears after scripts run is invisible to them, however complete the page looks in a browser. Guidance

An agent without credentials reads whatever an anonymous visitor gets. Gated content cannot be retrieved, so it cannot be cited. Guidance

A text-only reader gets nothing from an image without alt text. Information carried only by an image is lost when the page is converted to text. Guidance

robots.txt is where crawlers look first for what they may fetch and where the sitemap is. A server error on it can make crawlers treat the whole site as off-limits. Guidance

AI answer products with citations fetch through their own search crawlers (OAI-SearchBot, Claude-SearchBot, PerplexityBot). Blocking them, or blocking Googlebot and Bingbot, removes the page from those answers. Guidance

Being cited and being trained on are separate choices with separate crawlers. A site that has not named either leaves the decision to defaults, and an over-broad AI block can remove it from search too. Guidance

A sitemap lists every page you want found, with dates. Crawlers use it to discover pages and to decide what to recrawl. Guidance

A page missing from the sitemap depends on links alone to be discovered and recrawled. Guidance

lastmod tells crawlers what changed. When every URL carries the same date, the dates are being stamped at build time and crawlers learn to ignore them. Guidance

Crawlers and user-directed fetchers work to time budgets. A slow first byte risks a timeout before any content arrives. Guidance

The title and description are the summary a system reads before deciding whether the page answers the question. Guidance

The canonical tells crawlers which URL is the page. A missing or wrong canonical splits signals across duplicates, or points them at a different page entirely. Guidance

Open Graph tags are the card a link becomes when it is shared or previewed, including in chat interfaces. A broken image URL produces a broken card. Guidance

Pages without a device-width viewport render as shrunken desktop pages on phones, and mobile rendering is how search engines evaluate a page. Guidance

Structured data states facts about the page (what it is, who wrote it, when) in a form a machine does not have to infer. Its effect on AI citation specifically is contested, so treat it as clarity for machines rather than a ranking lever. Guidance

Organization and WebSite markup describe the site. A page-level type such as Article, WebPage or Product describes this page, and is what carries its headline, dates and author. Guidance

Markup that fails to parse is ignored. Markup that disagrees with the visible page (a different headline, an author as a bare string) gives a machine two versions of the facts. Guidance

Breadcrumbs place the page in the site hierarchy. Positions must run 1, 2, 3 with no gaps or the list is invalid. Guidance

FAQPage markup must describe questions a reader can see. Questions that exist only in the markup are the kind of mismatch the schema lesson warns against. Guidance

HTTPS is a baseline trust and ranking signal. An HTTP version that serves content instead of redirecting creates a duplicate of every page. Guidance

The H1 is the document title once markup is stripped. One clear H1 tells a parser what the page is about; none or several leaves it guessing. Guidance

Heading levels are the outline a text converter keeps. Skipped levels break the outline, so sections lose their relationship to each other. Guidance

main and article tell an extractor where the content starts and the navigation ends. Without them it has to guess what to keep. Guidance

A named person is something a reader, or a model, can check. "Admin" or "Team" gives nothing to verify. Guidance

An author page and sameAs links connect the name to a real, identifiable person, which is what lets the name carry any weight. Guidance

Freshness is one of the better-supported citation factors, and a machine can only judge it from dates it can parse. Visible dates and structured-data dates should agree. Guidance

Links to the sources behind a claim let a reader, or a system, check it. Whether the sources are primary still needs a human to judge. Guidance

Knowing who publishes a page, and how to reach them, is part of judging whether to trust it. Guidance