discoveryfiles.ai

The quick guide

Make your website useful to AI search.

Choose the questions that matter, publish useful answers and check how they appear. These five steps cover the SEO foundations needed for AI search.

Source review dates appear beside the advice. Open Details for fetch and publisher dates. About source dates.

01 · Choose your questions

Start with the questions your customers ask

Write down a few questions people ask before choosing or using your service. Use enquiries, support conversations and your own experience. Pick questions you can answer with facts from the business, and choose the search products your customers use. You do not need to track every assistant.

For each question, note the page that should answer it and what a useful result would be: understanding a limitation, finding a booking route or making a relevant enquiry. This gives you a small list of pages to work on and questions to revisit in step five. Save the current answers and citations before editing so you have a baseline for comparison.

Hypothetical example, used throughout: a bicycle repair shop in London wants customers to understand its e-bike repairs. It starts with “Who repairs e-bikes in London?” and “Does this shop repair my motor system?”, checking Google’s AI search features and Bing/Copilot.

Where SEO fits: search engine optimisation helps search engines discover and understand pages. Google and Bing describe their own AI search experiences as using established search foundations. You may see visibility work called GEO, or generative engine optimisation.

02 · Publish useful answers

Improve the page customers need

Use the page that already serves the question, or create one when there is a distinct need. Keep related answers together. A new page for every wording variation gives you more content to maintain without necessarily helping a reader.

  • Explain the offer and its limits. Say what you do, who it suits, where it is available and what is excluded. Include the next step, such as how to request an assessment or book.
  • Show what you know first-hand. Use a real example, original photograph, test result or documented process where it helps answer the question. Identify who did the work. State the conditions and limitations behind a result.
  • Make facts easy to check. Cite the source for a technical claim. Explain whether a figure is a measured result, an estimate or an example. Review facts when services, prices or availability change.
  • Make the page easy to use. Give it a descriptive title and clear headings. Check that the main answer is readable on mobile and that visitors can reach the relevant service and contact pages.

Our checklist applies Google’s and Bing’s guidance on original, focused, clearly structured content. It is an editorial recommendation, not a tested formula for citations.

Identify the business and provide working contact details. Explain legal and trading names where needed. See business identity and profile guidance for official records and listing requirements.

For the bicycle shop: improve its existing e-bike service page with the systems it actually handles, diagnostic limits and booking instructions. A dated repair example can demonstrate experience. Keep unsupported systems explicit so an answer can represent the service accurately.

03 · Check search access

Check that providers can use the page

Start with the page you just improved. Follow this sequence, then check your other key pages. If you use a managed website service, take the failed URL and the inspection message to the person who maintains it.

  1. Make the content reachable. Link to the page from relevant public pages. Check that its useful content loads without a login.
  2. Check crawl and indexing rules. Look for unintended robots.txt blocks, noindex in tags or response headers, and canonical URLs pointing elsewhere. Google’s AI search features also require indexing and snippet eligibility.
  3. Check firewall and CDN access. Confirm that intended, verified crawlers receive the useful page content, without a challenge or error. A successful status code alone is insufficient. Use the crawler reference for verification and firewall guidance.
  4. Check Google’s AI inclusion setting. In Search Console, open Settings → Search generative AI. Inclusion is the default; child properties inherit a parent’s setting unless overridden. Check the effective choice. This is separate from Googlebot crawling and Google-Extended’s content-use controls.
  5. Inspect in Google and Bing separately. Use URL Inspection in Search Console and Bing Webmaster Tools. Review the indexed result, then run a live test and inspect the returned content and response. After fixing a block, repeat the live test. Passing it does not establish indexing or an appearance in an AI answer.

Controls differ: OpenAI and Anthropic separate search and training crawlers. Perplexity says its search crawler and user fetcher do not collect foundation-model training data. User-requested access rules also differ by provider.

For the bicycle shop: inspect the e-bike page in Google and Bing. Check that the returned content includes supported systems and exclusions. If a firewall challenge appears instead, resolve that access problem before judging the page’s search visibility.

04 · Choose supporting formats

Add a format for a specific job

Check what your website software already provides. Keep one maintained source for each fact and generate supporting versions where practical. Choose formats supported by the tools you use.

FormatUse it forCheck
robots.txtCrawl rules for cooperating crawlers.Apply the provider’s documented rules. This is not authentication. Robots Exclusion Protocol.
XML sitemapListing URLs you want search engines to discover.List current, preferred URLs. The sitemap standard and Bing describe how sitemaps help URL discovery.
Structured dataDescribing page content for supported search features.Match visible facts: both Google and Bing require this. Google describes rich-result eligibility; Bing says markup may support grounding.

These formats support access, discovery or interpretation; they do not guarantee indexing or an appearance in an AI answer.

Optional llms.txt: the upstream proposal describes an index to useful content for compatible tools. Consider it when your tool documents support or you can test its use. Google says it does not help Search visibility or rankings, including its AI features. Google’s explanation.

For the format, a copyable generation prompt and dated updates, read the dedicated llms.txt guide.

For any added format, check the published response, the links and whether the intended tool uses it. Decide who will maintain it when the underlying pages change.

For the bicycle shop: check its existing sitemap and relevant markup against the updated service page. There is no need to add an llms.txt file unless the shop has identified a use for it.

05 · Check the results

Record appearances, accuracy and outcomes

Use the questions and starting record from step one. Make the change and note its date, check access and indexing, then repeat the same questions in the same products. Record whether the indexed page reflects the update. Repeat on a schedule you can maintain.

What to checkWhere to checkWhat to record
Access and indexingGoogle and Bing URL Inspection; verified crawler logs. Follow step three.Inspection date, status and whether the page reflects the update. A fetch alone proves neither indexing nor use in an answer.
Google appearancesGoogle’s Generative AI performance report.Link impressions and report period.
Bing appearancesBing’s AI Performance report.Sampled citations and report period. Coverage includes supported Microsoft AI experiences and selected partners.
Answer accuracyThe same questions with search enabled.Exact question, answer, cited URLs, product, model if shown, search setting and date; factual errors checked against your records.
Useful outcomesReferral analytics and enquiry or booking records.Identifiable visits and relevant enquiries or bookings; what the visitor needed. Referral data cannot explain every visit.

Keep impressions, citations and visits separate. Record unavailable data as unavailable, not as zero. These checks are a repeatable sample, not a complete visibility score.

For local questions, also record the product’s location context, language and any place named in the question. Use the local comparison checklist when testing across locations.

For the bicycle shop: repeat the two original questions and check its motor-system limits in the answers. If an answer is wrong, check the cited page before deciding what to correct.

Use what you find to choose the next small fix: remove an access block, correct a stale fact or answer a missing customer question. An improvement after an edit does not by itself prove that the edit caused it.