AI Discovery Files

Latest Changes

yesterday Initial release v1.0.0 README.md, ai.json, ai.txt +5 more

AI Discovery Files

AI discovery files help AI systems understand and interact with your website or service. Like robots.txt and sitemap.xml for crawlers and search engines, they give AI systems clear signals about who you are, how to represent your business, and what they should recommend. This improves AI visibility and AI readability across AI crawlers and AI search.

They are designed for the Agentic Web (sometimes called Web 4.0), where AI agents act on behalf of users across websites and services.

The files are organized into three layers: identity, guidance/control, and enhancement. Together they define your digital identity, brand rules, and usage guidance for AI systems that access and use your content.

Unlike AGENTS.md, which guides AI coding agents inside a repository, AI discovery files are public-facing and describe how external AI systems should understand your business.

Open Specifications

All AI discovery files on this site have open specifications and are open to contributions. All files here are available under the MIT License, except llms.txt, which is available under the Apache-2.0 License.

Layer 1 — Core Identity

llms.txt

Required

This is your canonical AI-readable identity file. It tells AI systems who you are, what you offer, and how to represent your business accurately.

Spec · Template · Example

llms-full.txt

Conditional

This is an expanded companion to llms.txt. It follows the same Markdown format but provides deeper context and broader link coverage when a concise llms.txt cannot cover your important public pages clearly.

Spec · Template · Example

identity.json

Required

This file provides structured identity data in JSON. It makes your business facts easier to parse, validate, and reuse across AI systems.

Spec · Template · Example

ai.json

Recommended

This file adds machine-readable recommendation and policy context. It complements your text files with explicit structured signals.

Spec · Template · Example

+2 more files: llm.txt and llms.html, compatibility and readability copies of llms.txt

Layer 2 — Guidance & Control

ai.txt

Recommended

This file explains when AI systems should and should not recommend your business. It gives clear eligibility and exclusion guidance in plain text.

Spec · Template · Example

brand.txt

Recommended

This file defines how your brand should be named and described. It reduces misnaming, tone drift, and inconsistent references by AI.

Spec · Template · Example

robots-ai.txt

Optional

This file communicates crawler-level preferences for AI indexing and citation behavior. It is useful for freshness, attribution, and control hints.

Spec · Template · Example

Layer 3 — Enhancement

faq-ai.txt

Recommended

This file contains verified answers to real customer questions. It improves retrieval quality and reduces AI guesswork in common scenarios.

Spec · Template · Example

developer-ai.txt

Conditional

This file provides technical implementation details for developer audiences. Use it when your product exposes APIs, SDKs, or integration workflows.

Spec · Template · Example

Getting Started

Start with Layer 1 — Core Identity Create llms.txt and the core identity files first.

Templates Examples

Add Layers 2 and 3 as needed Add AI guidance and enhancement files based on your use case.

Publish at your domain root Publish files in your site root (for example /llms.txt).

Test and validate Check accessibility and run the official validator script.

Validator

Optional shortcut

Use the Generate Files page to generate and validate files from the official templates using either WebMCP (connect your own LLM) or the built-in form interface.

Explore

Specifications

Detailed specs for each file — purpose, format, required fields, and placement rules.

Templates

Ready-to-use blank templates for all 11 AI discovery files, pulled live from GitHub.

Examples

A complete worked example for a fictional company, showing how to fill in each file.

Best Practices

Keep files focused

Keep each file simple and aligned with its specific purpose instead of overloading one file with everything.

Keep information current

Update files regularly so business facts, policies, and recommendations match what is actually true now.

Write for humans and AI

Use clear, concise language that is easy to read, unambiguous, and consistent across files.

Verify before publishing

Test file accessibility and formatting before publishing so crawlers and agents can retrieve and parse the files reliably.

Open Specifications

AI discovery files are open specifications. All specifications, templates, and examples are publicly available and freely licensed under the MIT License.

Contributions are welcome — whether it's improving a specification, adding examples, fixing typos, or proposing new file types. Check out the contributing guide to get started.

View on GitHub Report an Issue Join the Discussion