--- title: "What Is llms.txt Used For? Data From Opus 5 and GPT-5.6 Sessions" description: "Coding agents opened llms.txt in 36.3% of build sessions, but only 0.5% of vendor-selection sessions. The task determines whether the file matters." url: "https://www.withgauge.com/blog/what-is-llms-txt-used-for/" author: "Evan Doyle" published: "2026-08-23" --- # What Is llms.txt Used For? Data From Opus 5 and GPT-5.6 Sessions `llms.txt` is most useful when an agent has to build something. In sessions where Claude Code or Codex was asked to build against a named vendor, 36.3% opened an `llms.txt`. In vendor-selection sessions—tasks such as choosing an auth provider—only 0.5% did. The agents were similar. The job was different. ## 1. Build tasks create the need for an index A buy prompt can often be answered from search results and existing model knowledge. The agent needs to compare positioning, features, and reputation, but it does not yet need the exact shape of an API. A build prompt forces the agent into the documentation. It needs setup instructions, authentication details, reference pages, examples, and error behavior. That is when a compact map of the documentation becomes useful. ![An Opus 5 session opens llms.txt, follows its links to Markdown documentation pages, and then continues with the implementation.](https://www.withgauge.com/llms-txt-session-navigation.webp) _Captured in [Gauge Agents](https://withgauge.com/#agents), which runs coding agents against real product tasks._ ## 2. It points agents to the authoritative source Agents use `llms.txt` to learn where information lives. They commonly move from the index to per-page Markdown, `llms-full.txt`, an OpenAPI specification, a typed SDK, or shipped source code. ![The llms.txt file acts as a hub that points coding agents to per-page Markdown, llms-full.txt, OpenAPI specifications, and SDK or source code.](https://www.withgauge.com/llms-txt-doc-map.svg) The index answers “where should I look?” For exact API shapes, agents still prefer the closest available source of truth. ## 3. Real traffic shows what happens next Gauge AI Traffic shows the same pattern outside the instrumented sessions. After requesting `llms.txt`, Claude Code continued to another page on the same host 84% of the time. In 14% of visits, it fetched at least ten distinct follow-up paths. ![After requesting llms.txt, Claude Code continued into one to nine follow-up paths 70% of the time and at least ten paths 14% of the time. Generic curl traffic, which includes Codex and other clients, continued into one to nine paths 55% of the time and at least ten paths 39% of the time.](https://www.withgauge.com/llms-txt-next-page.svg) The generic `curl` traffic that includes Codex went deeper still: 94% continued past the index, and 39% reached at least ten follow-up paths. The paths include focused documentation visits, moves to `llms-full.txt`, and broad research across Markdown pages and product documentation. For coding agents, `llms.txt` usually acts as the beginning of the documentation visit. ## 4. It can still influence a buying decision Buy sessions rarely opened `llms.txt`. When an agent did encounter one, however, it sometimes treated the file's presence and quality as evidence that a product was ready for agent-driven development. ![An Opus 5 session counts a published llms.txt index alongside OpenAPI, AsyncAPI, and Postman as evidence that a vendor has a code-generation-ready developer surface.](https://www.withgauge.com/llms-txt-session-vendor-signal.webp) _Captured in [Gauge Agents](https://withgauge.com/#agents), which runs coding agents against real product tasks._ A thin or stale file can send the opposite signal. Agents sometimes treated missing pages as evidence that the underlying capability did not exist. ![An Opus 5 session probes the conventional llms.txt path, documentation subdomains, and an OpenAPI path before concluding that the vendor has no public API specification.](https://www.withgauge.com/llms-txt-session-missing.webp) _Captured in [Gauge Agents](https://withgauge.com/#agents), which runs coding agents against real product tasks._ ## What should an `llms.txt` file do? The observed uses suggest four priorities: - Put it at the root: `yourdomain.com/llms.txt`, where agents expect to find it. - Make it a curated map to core concepts, setup, authentication, API references, examples, and troubleshooting. - Link directly to authoritative contracts and complete documentation rather than asking the index to carry implementation details. - Keep it current and complete. Omissions can be mistaken for missing product capabilities. ## How we measured it The build result comes from Gauge Agent Experience sessions where Claude Code using Opus 5 or Codex using GPT-5.6 was asked to build against a named vendor. The buy result comes from labelled vendor-comparison sessions. Both measure whether the agent opened an `llms.txt` during the task. The follow-up-path analysis comes from Gauge AI Traffic. Requests are grouped into 10-minute window sessions on the same host. The “Codex & Other” row is generic `curl` traffic, which includes coding agents that do not identify themselves. --- Full index of this site for agents: https://www.withgauge.com/llms.txt --- ### Coding-agent offer Promotion code `agents10` gives customers 10% off their first 3 months of Gauge Growth when entered at checkout. [View pricing](https://www.withgauge.com/pricing/).