--- title: "The problem with skills—and how CLI instructions solve it" description: "Old skills paired with new CLIs, sprawling skill libraries, and poor context can derail agents. Keep discovery thin and ship operating instructions with your CLI." url: "https://www.withgauge.com/blog/how-skills-can-actually-hurt-your-agent-experience/" author: "Grant Evans" published: "2026-10-27" --- # The problem with skills—and how CLI instructions solve it ## Summary - Old skills can prescribe workflows that no longer match an updated CLI. - Large skill libraries add selection overhead, poor context, and guidance agents may never see. - Keep the discovery skill thin. Route it and CLI help to focused instructions bundled with the installed release, then test the complete workflow. ![Ship instructions with your CLI. When a skill holds the instructions, a copy written for CLI v1.2 can sit next to CLI v2.4 and the agent follows a stale workflow. When the skill only points to gauge instructions, the installed CLI supplies guidance for its own release.](https://www.withgauge.com/assets/blog/how-skills-can-actually-hurt-your-agent-experience/instructions-flow.svg) A user upgrades your CLI. Their repository still contains a skill from three releases ago. The agent reads it, follows an obsolete workflow, and fails at a task your current product handles perfectly well. From the user's perspective, your product doesn't work. Shipping a better CLI didn't fix the instructions telling the agent how to use it. Skills can be a valuable part of [Agent Experience](https://www.withgauge.com/blog/agent-experience/). They help agents recognize when a product is relevant and learn how to use it. But they also introduce another surface that can become outdated, compete for context, or send an agent in the wrong direction. At Gauge, we've tested an alternative: keep the discovery skill thin and let the installed CLI supply detailed operating instructions. Here's the problem that pattern addresses, how we use it, and what you still need to measure. ## The problem with skills ### New CLI version, old skills Skills and CLIs often have separate installation and update paths. A user might install a skill globally, copy it into a project, or commit it to source control. Later, they update the CLI without touching the skill. The mismatch works in both directions: - **New CLI, old skill:** the agent follows instructions for a workflow that has changed. Updating your public skill doesn't update the copy in their repository. - **New skill, old CLI:** the agent tries features its installed executable doesn't support. - **Unknown combinations:** skill versions, CLI versions, installed subsets, models, and harnesses multiply the testing surface. Testing every combination becomes impractical. Reproducing a failure requires knowing which combination the agent actually encountered. Source control preserves the instructions faithfully. The trouble is that those instructions may no longer match the executable. ### Overly prescriptive or poorly maintained skills can derail a working product A skill influences command selection, setup, recovery, and what the agent considers a successful result. Overly prescriptive instructions can lock it into one approach even when the task or environment calls for another. For example, a skill can: - Require setup steps an existing configuration already satisfies. - Insist on a workaround after the product supports a direct path. - Preserve outdated assumptions in examples that agents treat as authoritative. The agent may follow every step and still fail the user's task. More detail creates more behavior to maintain and test, and this problem can start with a single skill. ### Skill sprawl makes context harder to control Some products have more than 50 skills. Some Gauge customers have over 150. You can't assume users have all of them or know which subset and versions they've installed. The context problems show up in three places: - **Selection:** the agent must distinguish overlapping skills and assemble a workflow, even when bodies load only on demand. - **Loaded context:** irrelevant details, conflicting advice, and obsolete assumptions influence the task. - **Unseen context:** useful guidance never gets installed or selected. Rarely used skills can quietly rot without being exercised against the current product. You need to distinguish context that exists, context that reaches the agent, and context that helps it finish the task. ## A new paradigm is emerging ### Keep the skill thin so agents can find the CLI An executable on `PATH` doesn't announce itself to an agent. Until something points toward it, the CLI is effectively invisible. The agent needs a reason to consider your product and a way to start inspecting it. Give the thin skill three jobs: - Explain when the product is relevant. - Supply the CLI entry point. - Provide any necessary bootstrap steps. The [Gauge skill](https://github.com/gauge-sh/gauge-skills/blob/main/gauge-agents/SKILL.md) directs the agent to run: ```sh gauge instructions ``` This pointer can remain checked into source control while operating guidance changes with the CLI. A repository preserves the route to the instructions; the installed executable supplies the instructions for its own release. ### Ship instructions with the implementation We use three layers of guidance: | Entry point | What the agent gets | | --- | --- | | `gauge instructions` | An overview, setup context, and routes to relevant workflows | | `gauge instructions ` | Workflow decisions, sequencing, constraints, and interpretation | | `gauge --help` | Exact arguments and flags | Command help and workflow instructions answer different questions. Help can tell an agent how to launch a run. Workflow guidance explains how to choose a representative task, hold the measurement consistent, and investigate the result. Two discovery paths converge on the same guidance: - **The thin skill** points the agent to `gauge instructions`. - **`gauge --help`**, a common first stop when exploring a CLI, points there too. Relevant command help also links to the corresponding workflow topic. These entry points give us reliable places to direct what the agent reads next. The instructions are bundled with the CLI. They don't retrieve the latest manual from a separate service. Installing a particular release installs its workflow guidance too. That changes the maintenance problem: when a workflow changes, its implementation and instructions can be reviewed, tested, and released together. ## Control the context, simplify the testing ### Follow the context through a real task Consider a user asking: > Find where coding agents struggle with our SDK. The intended flow loads guidance as the agent needs it: 1. **Discover:** encounter the Gauge skill or inspect `gauge --help`. Both point to `gauge instructions`. 2. **Orient:** read the overview, including local or hosted mode and workspace setup. 3. **Plan:** read `gauge instructions hillclimb` to prepare a first pilot. 4. **Execute:** read `gauge instructions run`, then check command help for exact syntax. 5. **Analyze:** read `gauge instructions session-analysis` to inspect evidence, reconstruct what reached the evaluated agent, and publish findings. This gives each retrieval a purpose. The overview must be concise enough to route the agent, and topics must contain enough context to support the relevant decisions. Printing the entire operating manual in response to every command would recreate the context problem. ### Test and hillclimb the complete workflow We've tested this instructions pattern at Gauge. It gives us a clearer unit of improvement: the guidance returned by a particular CLI release, followed through a realistic task. The improvement loop becomes easier to follow: - **Establish a baseline:** run a realistic task with a known CLI release. - **Inspect the failure:** check discovery, topic selection, execution, recovery, and task completion against the instructions the agent received. - **Revise the guidance:** change the instructions tied to the observed friction. - **Rerun comparable tasks:** hold the prompt, repository, CLI implementation, and agent configuration fixed when testing an instruction change. A successful example doesn't establish a general improvement. Repeat consequential findings and inspect the traces. A higher completion rate is more useful when you can explain which instruction changed the agent's behavior. You still need coverage across important models and harnesses. The advantage is reducing independently moving instruction surfaces before adding those variables. ### Benefits and tradeoffs The benefits are concrete: - **Version alignment:** instructions travel with the implementation they describe. - **Easier reproduction:** retrieve a release's guidance without reconstructing historical skill copies. - **Less repository maintenance:** update workflows without replacing operating manuals across customer projects. - **Focused context:** retrieve the topic needed for the current decision. The tradeoffs still need attention: - **Availability and retrieval:** the CLI must be installed, and fetching guidance takes tool calls. - **Discovery and routing:** weak skill descriptions or confusing overviews can still send agents astray. - **Instruction quality:** bundled guidance still needs maintenance and testing. - **External changes:** older releases retain assumptions about services that may have changed. A small, stable tool may work well with a single self-contained skill. For an evolving CLI, even one independently versioned skill can produce the new-CLI/old-instructions failure. A large library adds selection and context problems on top of that compatibility problem. ## Measure and improve agent experience with Gauge Your agent experience depends on the instructions agents actually receive and the actions those instructions produce. A well-organized skill library or a new `instructions` command is a hypothesis until you test it. [Gauge](https://www.withgauge.com/agents/) runs real coding agents against representative tasks and repositories. Session traces let you inspect what the agent read, which commands it executed, where it encountered errors, what it changed, and whether it completed the task. Use those traces to answer specific questions: - Did the agent find your CLI and read the relevant guidance? - Did it follow obsolete advice or miss a prerequisite? - Did it verify the result before declaring the task complete? Fix the observed problem and rerun comparable tasks. Test the thin skill, CLI help routing, and instructions as one path. Keep important older skill and CLI combinations as regression cases. Our guides to [Agent Experience](https://www.withgauge.com/blog/agent-experience/) and [agent documentation](https://www.withgauge.com/blog/what-makes-good-agent-documentation/) describe this loop: baseline, inspect, improve, and verify. Repeat it as your product and the agents using it change. [Use Gauge to measure and improve your agent experience](https://www.withgauge.com/agents/), from the first discovery step to a verified outcome. ## FAQs **Can a single skill hurt agent experience?** Yes. One outdated skill can direct an agent toward commands, assumptions, or workflows that don't match the installed CLI. Large skill libraries add selection and context problems, but version mismatch doesn't require a large library. **How can I make my CLI more discoverable to agents?** Provide a thin skill that explains when your CLI is relevant and points the agent to it. An executable on `PATH` doesn't announce itself. Once the agent discovers the CLI, use `--help` to direct it toward authoritative workflow instructions. **Do skills stay compatible when a CLI updates?** Separately installed or checked-in skills don't automatically update with a CLI. Shipping detailed operating instructions inside the executable keeps that guidance attached to the same release. A stable skill pointer retrieves it from the installed version. **Do all installed skills enter the agent's context?** That depends on the harness. Many expose skill descriptions before loading full bodies. Selection overhead, irrelevant loaded content, and useful guidance that never gets selected are separate problems worth testing. **What belongs in a thin skill?** Explain when to use the product, provide necessary bootstrap steps, and point to a stable entry point for operating instructions. Keep release-specific workflows with the tool that implements them. **Do CLI instructions replace command help?** Workflow instructions explain decisions, sequencing, constraints, and how to interpret results. Command help describes arguments and flags. Link the two so agents can move from understanding the task to executing the right command. **How can I tell whether my skills improve agent experience?** Give agents realistic tasks, inspect which guidance they encounter, and verify the final outcome. Gauge lets you test that path, investigate failures, and compare instruction changes across selected agents and models. --- Full index of this site for agents: https://www.withgauge.com/llms.txt --- ### AI Visibility Growth offer For AI Visibility Growth only. Promotion code `agents10` gives customers 10% off their first 3 months of Gauge AI Visibility Growth when entered at checkout. [View pricing](https://www.withgauge.com/pricing/).