ai.txt: Two Competing Standards, One Filename

Search for “ai.txt” and you will find two sets of instructions that contradict each other. Neither is wrong. They are two entirely unrelated specifications that happened to claim the same filename, and the confusion is now baked into the ecosystem.

Standard 1: Spawning's ai.txt

Spawning built ai.txt as a text-and-data-mining opt-out, aimed squarely at artists and publishers who wanted their work excluded from training sets. It borrows robots.txt syntax wholesale:

User-Agent: *
Disallow: images
Disallow: video

User-Agent: GPTBot
Disallow: /

The one genuine addition over robots.txt is media categories. Alongside URL paths, a rule value may be images, text, audio, video or code — so a photographer can opt images out while leaving written posts available. robots.txt cannot express that.

The context that made this matter is legal, not technical: EU copyright law lets rightsholders reserve TDM rights, but only if the reservation is expressed in a machine-readable way. A file like this is one attempt at that expression.

The honest caveat: adoption is thin. The major AI vendors document robots.txt tokens and honour those. None of them publicly commits to reading ai.txt. As a statement of intent it has value; as an enforcement mechanism it is weaker than one Disallow line in the file crawlers actually read.

Standard 2: aitxt.ing's ai.txt

Completely different problem. This one is not about restricting anything — it is about correcting what AI assistants say about you.

If you have ever watched a chatbot confidently tell a user that your product has a free tier it does not have, or an API you never shipped, you know the failure. The aitxt.ing format is a short Markdown document, optionally with YAML frontmatter, that states plainly what your product is and — crucially — what it is not:

---
updated: 2026-09-14
scope: /
---

# Acme Analytics

Privacy-first web analytics. Self-hosted or cloud.

## What we offer
- Cloud plans from $9/month
- Self-hosted community edition, free, MIT licensed

## What we do NOT offer
- No free cloud tier. The free option is self-hosting only.
- No session recording or heatmaps.
- No mobile SDK.

Three frontmatter keys are defined:

• updated — an ISO 8601 date, so a consumer can judge staleness.
• scope — the absolute path this file describes.
• parent — a link to a broader file, for cascading coverage.

The “what we do NOT offer” section does the heavy lifting. Models hallucinate plausible features; an explicit negative is far more effective at suppressing that than any amount of positive description. Keep the whole file under about 10 KB — it is a correction sheet, not a documentation dump, and length dilutes the signal.

Side by side

So which do you publish?

Work backwards from the problem, and notice that the two ai.txt formats are not really each other's alternatives.

• Want AI to stop training on your work? Use robots.txt with the per-vendor tokens. Add a Spawning ai.txt as a supplementary statement of intent if you want the record, but do not rely on it alone.
• Want AI to stop inventing features you do not have? Publish an aitxt.ing file. Nothing else does this job.
• Want AI to find and cite your best content? That is llms.txt, a third thing entirely.

Most sites end up with robots.txt plus llms.txt, and add an aitxt.ing file when they have a product story that keeps getting told wrong. Only one file can live at /ai.txt, so if you publish one, pick deliberately — and expect some consumers to guess wrong about which format it is.

Validating either one

Our validator supports both, as separate formats with separate rule sets — merging them would mean checking every file against rules written for a different standard. Detection uses the same three-second test described above, and you can override it from the dropdown.

One check worth calling out: if you paste an aitxt.ing file containing a stray User-Agent: line, the validator warns you. That is almost always someone who read the wrong specification halfway through.