Learning Path

Navigate the AXO curriculum

Learning paths for websites, applications, and AXD.

AXO Quick Start Guide

Implement Agent Experience Optimization in 10 practical steps with ready-to-use code templates and checklists.

What You'll Accomplish

Foundation

Set up basic structure and markup

Enhancement

Add trust signals and structured data

Optimization

Monitor and iterate improvements

Prefer to limit or block AI crawlers?

Wondering whether this is called GEO, AEO or something else? The naming is still disputed and matters less than the ten steps below.

If your goal is to prevent AI training or referencing, see our AI Opt-Out Controls guide for robots.txt and ai.txt templates, plus server-side controls and monitoring tips.

Add Basic JSON-LD Schema

Start with Article or WebPage schema to help agents understand your content structure.

Article Schema Template

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "Article",
  "headline": "Your Article Title",
  "author": {
    "@type": "Person",
    "name": "Author Name"
  },
  "datePublished": "2026-01-15",
  "dateModified": "2026-01-15",
  "description": "Article description"
}
</script>

Create robots.txt with AI User-Agents

Configure crawler access with common AI agent user-agents.

robots.txt Template

User-agent: *
Allow: /

# Training crawlers
User-agent: GPTBot
Allow: /

User-agent: ClaudeBot
Allow: /

# Search/citation crawlers — these decide whether you can
# appear in AI answers, so treat them separately from training
User-agent: OAI-SearchBot
Allow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: PerplexityBot
Allow: /

# Google's AI training/grounding opt-out token.
# (There has never been a "Bard" user-agent — the token is Google-Extended.)
User-agent: Google-Extended
Allow: /

Sitemap: https://yoursite.com/sitemap.xml

Implement llms.txt (Emerging Standard)

Add llms.txt as a curated, markdown-native map of your site for agents and developer tooling.

⚠️ Set your expectations first: llms.txt does not drive citations

• Ahrefs studied 137,000 sites and found 97% of valid llms.txt files received zero requests in May 2026 (Ahrefs, 2026).
• Google has publicly documented that it does not use llms.txt for Search or AI Overviews.
• Its real value today is as agent and developer tooling—Google added llms.txt to Lighthouse's agentic-browsing audits.
• Treat it as a nice-to-have after your HTML, schema, and sitemap are right. It is not a ranking or citation lever.

📋 llms.txt vs Other Standards

• llms.txt: Proposal for a markdown site map aimed at agents
• robots.txt: Established crawler control standard
• sitemap.xml: Content discovery for search engines
• ai.txt: Alternative proposal with similar goals

llms.txt Template (llmstxt.org format)

# Your Site Name

> One or two sentences describing what this site covers and who
> it is for. This blockquote summary is part of the spec.

Optional prose here: scope, conventions, anything an agent needs
before it starts following links. No headings in this part.

## Docs

- [Getting Started](https://yoursite.com/docs/start.md): Install and first run
- [API Reference](https://yoursite.com/docs/api.md): Every endpoint with examples

## Guides

- [Schema Markup](https://yoursite.com/guides/schema.md): JSON-LD patterns
- [Measurement](https://yoursite.com/guides/measurement.md): Tracking agent traffic

## Optional

- [Changelog](https://yoursite.com/changelog.md): Release history

The format is fixed: a single H1 title, a > blockquote summary, optional prose, then ## sections containing markdown link lists in the form - [name](url): description. A section literally named ## Optional marks links an agent may skip when short on context. Free-form key/value lines are not part of the spec.

Optimize Heading Structure

Ensure proper H1-H6 hierarchy for predictable content chunking.

Single H1 per page

Logical H2-H6 nesting

Descriptive heading text

Add Author and Date Information

Include clear authorship and publication dates for trust signals.

<article>
  <header>
    <h1>Article Title</h1>
    <div class="article-meta">
      <span class="author">By John Smith</span>
      <time datetime="2026-01-15">January 15, 2026</time>
      <time datetime="2026-01-20" class="updated">
        Updated: January 20, 2026
      </time>
    </div>
  </header>
  <!-- Article content -->
</article>

Create FAQ Schema

Add FAQ structured data for common questions agents might ask.

{
  "@context": "https://schema.org",
  "@type": "FAQPage",
  "mainEntity": [{
    "@type": "Question",
    "name": "What is Agent Experience Optimization?",
    "acceptedAnswer": {
      "@type": "Answer",
      "text": "AXO is the practice of optimizing content..."
    }
  }]
}

Update Sitemap with lastmod

Ensure your sitemap includes accurate last modification dates.

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://yoursite.com/article</loc>
    <lastmod>2026-01-15T10:30:00Z</lastmod>
    <changefreq>weekly</changefreq>
    <priority>0.8</priority>
  </url>
</urlset>

Set Up Agent Traffic Monitoring

Track AI agent crawls in your analytics to measure engagement.

🤖 Common AI Agent User-Agents to Track

• GPTBot - OpenAI training crawler
• OAI-SearchBot - ChatGPT search index (drives citations)
• ChatGPT-User / OAI-AdsBot - ChatGPT browsing and ads
• ClaudeBot / Claude-SearchBot / Claude-User - Anthropic crawler, search index, user fetch
• Google-Extended - Google's AI training/grounding opt-out token (there is no Bard user-agent)
• Google-Agent - Google user-triggered fetcher, added March 2026; ignores robots.txt by design
• PerplexityBot / Perplexity-User - Perplexity index and user fetch
• Applebot-Extended, meta-externalagent, Amazonbot, Bytespider, CCBot, MistralAI-User, DuckAssistBot

Validate Implementation

Use validation tools to check your schema markup and structure.

Schema Validation

Schema.org Validator

HTML Validation

W3C Validator

Monitor and Iterate

Set up ongoing monitoring and improvement processes.

Weekly crawl rate monitoring

Monthly citation tracking

Quarterly content audits