Generate an llms.txt

llms.txt is an emerging standard that lets you tell AI crawlers which pages on your site are most important and how they're structured. It's not yet officially recognized by every model, but ChatGPT, Claude and a growing list of LLM crawlers now respect it. Generating one takes about 10 minutes and there's no downside.

What llms.txt looks like

It's a single Markdown file at your domain root. Roughly:

text
# SEMOptimiser

> The all-in-one SEO, analytics and audit platform for teams who play to win search.

## Docs

- [Quickstart](/docs/quickstart): set up your first audit in two minutes.
- [API reference](/api-reference): REST API + webhooks.
- [Pricing](/pricing): plan comparison.

## Blog

- [The 2026 AI Visibility Playbook](/blog/ai-visibility-2026): 47-point guide.
- [Core Web Vitals INP migration](/blog/core-web-vitals-inp): 30-day plan.

Generate it automatically

1. Go to AI Visibility → llms.txt → Generate.
2. We crawl your site, group pages by section, and propose a llms.txt with your top pages auto-selected.
3. Review the proposal. Drag to reorder, remove pages, or add a short description to any entry.
4. Click Download or, if you've connected a server, click Deploy to push it to your domain root.

Hosting the file

The file must be reachable at https://yourdomain.com/llms.txt – the same way robots.txt sits at /robots.txt. If you can't deploy via our server connection, your CMS can usually host it as a static file or a custom route.

WordPress

Upload via the WP file manager (or SFTP) directly to the WordPress root, alongside wp-config.php. Don't use a posts/pages system – it needs to live at the root.

Next.js / Nuxt / Astro

Drop the file into your /public directory. It will be served at the root in production.

Static hosting (Vercel, Netlify, S3+CloudFront)

Same as Next.js – drop it in your public/static directory and redeploy.

Keeping it in sync

If you connected a server during generation, we re-deploy your llms.txt automatically whenever your sitemap changes meaningfully (e.g. a new top-priority page is published). Toggle this behaviour off in AI Visibility → llms.txt → Settings if you prefer manual control.

How much it actually moves the needle

Honest answer: it's early. Sites that have published llms.txt for 6+ months report 20-40% higher citation rates in ChatGPT and Claude compared to comparable sites without one. Gemini and Perplexity show smaller but positive lifts. We don't track Bing-only LLMs because Bing itself crawls everything anyway.