Selling to coding agents instead?Go to Gauge Agents
Resources

11 minSeptember 3, 2026Author:Farbod MemarianFarbod Memarian

TL;DR

  • Open-source coding agents require a distinct Agent Led Growth strategy because developers can combine models such as DeepSeek, Qwen, GLM, and Minimax with different open harnesses.

  • Model knowledge likely carries more relative weight for open source agents, since open-weight models tend to have thinner, less funded training coverage of niche developer tools than closed frontier models.

  • Clear docs, GitHub READMEs, comparison pages, and forum discussions help models place your product in the right category.

  • Fresh docs and llms.txt files help agents verify current capabilities during live research instead of treating missing information as a missing feature.

  • Manual tracking breaks down across many models and harnesses. Gauge runs real coding tasks and measures pick rate, mention rate, and install rate across open and closed agents.

Why open-source coding agents require separate analysis

Open source coding agents deserve their own lane within Agent Led Growth because developers can change the model, harness, and deployment independently. A developer might run DeepSeek through OpenCode on local infrastructure, then switch to Qwen for the same repository. Developers may make these swaps to change cost, deployment control, or model behavior.

Agent Led Growth covers two separate problems. Agent Preference Optimization, or APO, focuses on whether a coding agent chooses your product. Agent Experience, or AX, focuses on whether the agent installs and configures your product correctly after choosing it. This guide focuses on APO for open source agents.

Fail APO and you are never in the running. Fail AX and you get picked, break during setup, and get replaced before anyone reviews the diff.

Closed products such as Claude Code and Codex give their vendors more control over the model and runtime. Open source deployments vary more. A developer can self-host GLM or Minimax, connect either model to an open harness, and change which search or terminal tools the agent can use.

Each model and harness pairing can recommend different tools for the same task. The model brings its existing knowledge and preferences, while the harness controls what context and research tools it can access. You therefore need to evaluate recommendation behavior across the open source lane rather than treating every coding agent as another version of Claude Code.

That evaluation is the hard part, since none of it shows up in your normal analytics. Gauge exists to make this lane visible by running real coding agents, open and closed, against real tasks and recording what each one actually does.

Why model knowledge matters more when the model has seen less

Open source coding agents choose tools using three inputs. Gauge's own research on Agent Preference Optimization treats this three-input pipeline as the standard way to explain how any coding agent decides, closed or open.

  • Model knowledge — what the model learned during training.

  • Session context — the prompt, repository, rules files, and user preferences.

  • Live research — information gathered through search, documentation, and source code during the task.

Model knowledge often shapes the candidate list before research begins. This is where open source models likely diverge most from closed frontier models. Labs like OpenAI, Anthropic, and Google spend heavily to crawl and curate the web for training, while open-weight labs generally work with smaller budgets and less complete coverage. Gauge has not measured the training data of DeepSeek, Qwen, GLM, or Minimax directly, but that gap in resourcing is a reasonable basis to expect thinner, less consistent knowledge about niche developer tools than a heavily funded closed model carries. When an open model knows only a few tools in a category, those familiar names become its starting options. Session context can narrow the choice, but it cannot reliably introduce a product the model does not know.

Live research can expand the candidate list, but the agent may never search broadly. An open harness might browse only after encountering missing information, or it might use search to confirm a tool already recalled from training. The model's initial candidate list can therefore determine which products receive further evaluation.

A short candidate list makes unclear product positioning more costly. If training content describes your product inconsistently, the model may place it in the wrong category or fail to connect it to the task. A richer model may have enough separate references to resolve the ambiguity. A model with limited information may rely heavily on a README, outdated documentation, or a forum discussion.

You can shape that prior by publishing consistent, explicit descriptions across the sources models learn from. Your docs, GitHub README, comparison pages, and forum presence should state what the product does, which category it belongs to, and when an agent should choose it.

Each of the three inputs runs on a different clock, and that changes where you should invest. Model knowledge is the hardest to influence and only gets re-rolled when a new version ships, but the effect lasts the longest once you are in it. Session context resets every session and is mostly out of your hands. Live research is the fastest surface to move and runs fresh on every query, which is exactly why it gets most of the attention in general AEO advice. For open source agents with thinner priors, that balance shifts: the slow-moving, hard-to-influence model knowledge stage carries more relative weight than it does for a closed frontier model with a richer prior to fall back on.

Publish product information where models and agents can find it

Public technical content gives open source coding agents the raw material for understanding your product. Exact training sets remain opaque, but docs, GitHub READMEs, comparison pages, Reddit, and open forums can shape a model’s prior. A model with limited evidence about niche tools may misclassify your product or ignore it.

This is not a guess about what agents value. Across 500 observed coding-agent runs, page fetches broke down like this:

  • Documentation — 55%

  • Source code — 18%

  • Package registries — 11%

  • Third-party content — 5%

Inside those documentation fetches, setup pages, READMEs, and quickstarts alone made up nearly 60%. Agents read to execute, not to be persuaded, so the blog posts and listicles built to win over a human reader carry far less weight than a clear setup guide.

Your README should define the product before explaining how to install it. Name the category in plain language, describe the main use case, and use the same product and package names found in your docs. Avoid taglines that require background knowledge. An agent should understand what the product does after reading a short excerpt without the rest of the page.

Your docs should repeat the same category positioning while adding concrete technical details. Installation pages should name supported languages or frameworks, while integration guides should explain when someone would use the product. Consistent wording helps a model connect the company, product, package, and use case instead of treating them as unrelated concepts.

Comparison content teaches models which alternatives belong in the same decision set. Publish accurate pages that explain where your product fits and how its tradeoffs differ from named alternatives. You should also give independent writers clear technical facts they can verify. Vague claims such as “developer-first infrastructure” give a thinner model little information about when to recommend you.

Reddit and open forums can reveal the terms and questions users apply to your product. Answer technical questions directly, correct outdated information, and help users solve real problems without pretending to be an independent customer. Agents can encounter these discussions in training data or live research, so inaccurate old threads may keep influencing how they describe your product.

Treat every public page or profile as a product reference that an agent may retrieve. Audit whether each one states the same category, current package name, and supported use case. Training snapshots may lag behind your edits, but clear and consistent updates improve the material available to future models and agents that research the web during a task.

Keep documentation and llms.txt current for live research

Live research can override a favorable training-time impression when an open harness finds incomplete documentation. An agent may know your product fits the task, then drop it after your docs fail to confirm a required feature, package, or integration.

When an agent fetches llms.txt, an incomplete file can keep relevant documentation out of its research path. Gauge found that agents opened llms.txt in 36.3% of named-vendor build sessions, compared with 0.5% of vendor-selection sessions. In the same Gauge analysis, Claude Code continued to another page on the same host in 84% of sessions where it fetched the file. If llms.txt omits an authentication guide or API reference, an agent that relies on the file may fail to find evidence that the product supports the required task.

Keep llms.txt at the root of your domain and link it to the authoritative pages an agent needs to complete a build. It should cover:

  • Setup

  • Authentication

  • API references

  • Examples

  • Troubleshooting

Update the index whenever you add a product capability, rename a package, change an install command, or move a documentation page.

Each linked page also needs fresh facts in a format agents can read cleanly. Gauge recommends putting the current package version, exact install command, and recent API changes near the top of the page. Serve a Markdown version when possible, and make sure its facts match the public HTML page. A link to a docs map cut average missing-page requests from 2.23 to 0.11 per task in Mintlify's test of 2,400 agent runs, which shows how much a thin index costs you in wasted fetches and dead ends.

Named-agent traffic also understates how much of this is happening. Codex has no general-purpose fetch tool, so it opens known URLs like llms.txt through a shell curl command instead, and the server logs that as generic curl rather than Codex. In Gauge's own data, roughly half of llms.txt-reading runs used a request path that never self-identified. Open harnesses built the same way will likely hide from your analytics in the same manner, so a dashboard of named AI user agents is probably undercounting the open source traffic actually hitting your docs.

Open-web content may contribute to what a model already knows, while current documentation gives web-enabled agents evidence they can verify during a task. Review both separately, especially after releases that change setup, authentication, package names, or supported integrations.

Why tracking one open-source agent at a time does not scale

Manual checks cannot represent a fast-changing mix of open models and harnesses. New DeepSeek, Qwen, GLM, and Minimax versions can behave differently, while updates to harnesses such as OpenCode and Pi can change the tools and context available during a task. A Qwen result in OpenCode may not predict how another Qwen version behaves in a different harness.

You need repeated runs to measure recommendation behavior, tracked across three metrics:

  • Mention rate — how often the agent considers or names your product.

  • Pick rate — how often the agent chooses your product when it fits the task.

  • Install rate — whether the agent moves past consideration and adds the package.

A single prompt test captures one decision under one set of conditions. The outcome may change with the repository, task wording, model version, or available context. Manual spot checks also hide cases where an agent mentions your product but installs a competitor.

The testing load grows with every model and harness you add. New releases can invalidate earlier findings, so an occasional spreadsheet check quickly falls behind. You need a recurring test suite that runs the same tasks across relevant agents, records their decisions, and preserves enough detail to explain why each choice happened. A recurring test suite lets you track results, correct weak positioning or documentation, and test the same conditions again.

Run a repeatable measurement loop

A useful ALG program repeats the same tasks under controlled conditions and records which product each agent mentions, picks, and installs. Inspect the session trace to identify the retrieved pages or model assumptions that shaped the choice. After correcting a specific positioning or documentation problem, run the same prompt against the same repository again.

Re-running the task shows whether pick rate changed after the fix. Install rate reveals whether agents moved beyond a recommendation, while mention rate shows whether your product entered consideration at all. Repeated runs help separate a real change in behavior from normal variation between sessions.

Gauge runs real coding agents against real tasks in isolated repositories. It records searches, fetched pages, package installs, file changes, and implementation outcomes. Gauge covers DeepSeek, Qwen, GLM, Minimax, OpenCode, and Pi alongside closed models, so you can compare behavior without testing every agent and harness by hand.

Gauge can also replay a session from the exact point an agent read your documentation, swap in the fixed page, and rerun it with everything else held constant. That isolates whether your fix actually caused a different outcome instead of just coinciding with normal variation between runs.

Open models, harnesses, and documentation keep changing. You should keep the prompt and repository stable, review where each run failed, make one clear fix, and test again. That loop connects documentation and positioning work to measured changes in pick rate on the agents you care about.

FAQs

How do open source coding agents differ from Claude Code or Codex?

Open-source coding agents combine a model with a configurable harness or self-hosted environment, while Claude Code and Codex provide more integrated configurations. Gauge compares how DeepSeek, Qwen, GLM, Minimax, and other agents recommend the same products. You can see where each model’s knowledge and research behavior produce different picks.

Does llms.txt improve search visibility or citation rates?

An llms.txt file is a documentation index that some coding agents fetch during build sessions. It is not a standard search-engine ranking or citation mechanism. Gauge tracks when agents fetch llms.txt and which documentation pages they visit next. A fresh index helps agents find current setup instructions, API references, and troubleshooting pages.

How should I prioritize agents when I cannot track every one manually?

Prioritize the model and harness combinations your users are most likely to run, then compare their pick rate, mention rate, and install rate. Gauge runs recurring tasks across open source and closed agents, then compares their behavior in one place. You can focus fixes on the agents where clearer positioning or documentation can change recommendation behavior.