In Practice
The part after you have read the specification
The specifications tell you what a valid file contains. They do not tell you how to serve it on Shopify, which crawler you are actually blocking, or what to do when your site is in three languages. That is what this section is for.
For AI agents authoring these files
The specifications say what a valid file contains. This says how an agent should produce one: what it may infer, what it must ask the owner, and a nine-step procedure. Machine-readable at /agent/authoring.md.
ReadPublishing the files on your platform
Where the files go, how to serve the right Content-Type, and how to do the llm.txt redirect on Cloudflare Pages, Netlify, Vercel, nginx, Apache, and hosts that cannot redirect at all.
ReadThe AI crawler table
Every documented AI crawler, who runs it, whether it trains or searches, and what you actually give up by blocking it. Sourced from each operator’s own documentation.
ReadBeyond the files
Other levers on how AI systems see you: Google’s preferred sources button, Organization structured data, IndexNow, and the identity hygiene that matters more than any of them.
ReadMultilingual sites
Publishing language variants at language paths, what the root file links to, and when a single file is the better answer.
ReadThese are implementation guides. For what a valid file must contain, the specification documents in the project repository are the authority.
Found something here that is wrong or out of date? Email [email protected]. Platform behaviour changes and we would rather fix it than leave it.