Your next user may be someone's AI agent. Testing whether it can use your product is a new job.
Your next user may be someone's coding agent, and if it gets stuck on your install step, you lose that user without ever knowing. Microsoft and Gauge both open-sourced tools this month that send real agents at a product and record where they fail. Gauge's hosted plans run from $100 to $20,000 a month (its listed pricing), which leaves room for a fixed-fee audit.
Published Oct 21, 2026
The Wow
What someone built
Gauge's open-source CLI runs coding agents in Docker against a product, captures each session and reports issues such as interactive commands that block agents, deprecated commands and unused skills (README and Show HN post, Oct 20, 2026). Microsoft's Scope repository evaluates how agents use a product against reusable success criteria, with each run's evidence inspectable.
- Made by
- Gauge (gauge-sh on GitHub) (not us)
- Original repository
- Gauge CLI repository
- License
- MIT for Gauge CLI and for Microsoft Scope, per their repositories
- What you need
- Node.js 22.13+ and npm
- Docker running Linux containers
- A login or API key for the coding agent you test (Claude Code, Codex, Pi or OpenCode, per the README); usage is billed by your provider
- Install
- Gauge: install the CLI from npm (@withgauge/cli) and point your coding agent at the repo, per its README. Scope: follow the getting-started docs linked from microsoft/scope.
- A business you could build on it
- Fixed-fee agent experience audit for small developer-tool startups
Why it's remarkable
Usability testing has a new kind of user. Two tools arrived within weeks of each other, and companies such as PostHog and Clerk appear on Gauge's customer list, so the category already has buyers at the top end.
The Opportunity
The business it reveals
A fixed-fee agent-experience audit for startups too small for a monthly platform: run realistic onboarding tasks with two or three agents, find where each gets stuck, and hand over the fixes.
Who would pay
Seed and Series A API, SDK and developer-tool startups with a docs site and a CLI.
The Flip
3 ways you could turn this into a business
Fixed-fee agent experience audit for small developer-tool startups
- Who pays
- Seed and Series A API, SDK and CLI companies
- How it would earn
- One-time audit fee, with an optional re-test after fixes (hypothesis; price untested)
- Why it might work
- The category already has paying customers at the top end, and the free tools lower the cost of serving smaller teams.
- What already exists
- Gauge's hosted Agents plans run from $100 to $20,000 a month (according to its pricing page); Microsoft Scope is free but a research preview; developer-relations and docs consultancies offer broader reviews.
Agent-ready docs rewrite subscription
- Who pays
- Developer-tool companies with fast-changing docs
- How it would earn
- Monthly retainer to keep docs, error messages and skills agent-friendly (hypothesis)
- Why it might work
- Gauge's README lists outdated commands and confusing terminology as common failures, which docs work fixes.
- What already exists
- Docs platforms and technical-writing agencies already serve this market; the agent-testing loop is the differentiator.
Agent-shopping readiness checks for online stores
- Who pays
- Small e-commerce brands
- How it would earn
- One-time check plus fixes (hypothesis)
- Why it might work
- If shoppers send agents to compare and buy, a store whose pages confuse agents may lose sales.
- What already exists
- Not researched in depth. Commerce platforms may build this into their own products, which would make it a feature rather than a service.
Why we'd try "Fixed-fee agent experience audit for small developer-tool startups" first: Paying demand is visible at the top of the market, the tools are free, and a single failure found in a founder's own onboarding is a strong first pitch.
Build It
How you could build it
A five-day audit package: task list, agent runs, failure report, fixes, and a re-run.
First step: Install Gauge CLI, pick a developer tool you already use, and run one onboarding task with two coding agents. Write down the first place each agent got stuck.
Agents and tools
- Gauge CLI
- Microsoft Scope (optional)
- Docker
- Claude Code, Codex or Cursor subscriptions
Setup
- Write a reusable list of 10 onboarding tasks (install, auth, first call, common error).
- Run the tasks on two public tools to build sample reports.
- Create a one-page report template with before/after results.
- Set a fixed audit scope and turnaround.
What the agent does
- Runs every task across several agents
- Captures transcripts and failure points
- Drafts candidate fixes to docs and error messages
- Re-runs tasks after fixes
What you do
- Chooses the tasks with the client
- Judges which failures matter
- Writes the final recommendations
- Sells and presents the audit
Your first customer: A small developer-tool founder you contact with one real failure you found in their onboarding.
Test demand first: Send three founders one real failure each with a proposed fix. Success: at least one asks for the full audit.
Your starter instructions
Paste this into the AI agent you use (ChatGPT, Claude, Gemini, Grok Bot or another) as your first message. Copying it starts work in your own agent; it doesn't create or deploy a business by itself.
Running it
What stays human: Choosing realistic tasks with the client; judging which failures matter; writing the fix recommendations; presenting results; never changing the client's code without approval.
- Startup cost (estimate)
- About $0 plus agent subscriptions you may already have (Estimate: both tools are open source and run locally with your own agent plans, per their READMEs. Not measured)
- Running cost
- Agent usage per audit is unknown; Gauge's hosted product quotes about $3 to $4 per eval (according to its pricing page), which is a reference, not our cost.
- How much the agent runs
- People run it; AI helps with tasks.
Pricing your own version? Use the free startup-cost worksheet.
The evidence
What the source shows: The projects' own repositories and launch post. We read them; we haven't run a client audit.
What's ours: The fixed-fee audit is our application of the free tools.
Worth planning around
- The free tools could let a startup do this in-house.
- Agent behavior changes with each model version, so findings go stale quickly; date every report.
Sources
- Gauge CLI repository (gauge-sh/gauge-cli), README and MIT license
- Gauge on GitHub
- Microsoft Scope repository (microsoft/scope)
- Show HN: Gauge CLI (Oct 20, 2026)
- Gauge: pricing and customers
Sources reopened Oct 20, 2026. Spot a mistake? Tell us and we'll correct it in public.
Keep reading
- Services to sell to people building apps with AI (guide)
- Cursor: plans and what people build and sell with it (guide)
- How much of the work do AI agents actually run? (guide)
- He still can't write code. His five-person ad agency now runs on about 270 tools he built with Claude. Every owner-run firm needs its first five
- A marketer who doesn't code built an elk-call coaching app with Claude Code and says it hit a $650K run rate in 110 days. Every skill learned by sound is the same opening
- A free n8n workflow flags fake supplier invoices before anyone pays them — and Germany's e-invoice switch makes it a service bookkeepers can sell
- Marc Lou's agent chased a broken-shoe refund through three support desks and got his 850 AED back. Small stores now need a claims desk that can answer agents
- Every discovery and launch plan