Open-Source Tool

Your next user may be someone's AI agent. Testing whether it can use your product is a new job.

Your next user may be someone's coding agent, and if it gets stuck on your install step, you lose that user without ever knowing. Microsoft and Gauge both open-sourced tools this month that send real agents at a product and record where they fail. Gauge's hosted plans run from $100 to $20,000 a month (its listed pricing), which leaves room for a fixed-fee audit.

Published Oct 21, 2026

The Wow

What someone built

Gauge's open-source CLI runs coding agents in Docker against a product, captures each session and reports issues such as interactive commands that block agents, deprecated commands and unused skills (README and Show HN post, Oct 20, 2026). Microsoft's Scope repository evaluates how agents use a product against reusable success criteria, with each run's evidence inspectable.

Open-Source Tool

Original repository
Gauge CLI repository
License
MIT for Gauge CLI and for Microsoft Scope, per their repositories
What you need
  • Node.js 22.13+ and npm
  • Docker running Linux containers
  • A login or API key for the coding agent you test (Claude Code, Codex, Pi or OpenCode, per the README); usage is billed by your provider
Install
Gauge: install the CLI from npm (@withgauge/cli) and point your coding agent at the repo, per its README. Scope: follow the getting-started docs linked from microsoft/scope.

Why it's remarkable

Usability testing has a new kind of user. Two tools arrived within weeks of each other, and companies such as PostHog and Clerk appear on Gauge's customer list, so the category already has buyers at the top end.

The Opportunity

The business it reveals

A fixed-fee agent-experience audit for startups too small for a monthly platform: run realistic onboarding tasks with two or three agents, find where each gets stuck, and hand over the fixes.

Who would pay

Seed and Series A API, SDK and developer-tool startups with a docs site and a CLI.

The Flip

3 ways you could turn this into a business

Agent-ready docs rewrite subscription

Who pays
Developer-tool companies with fast-changing docs
How it would earn
Monthly retainer to keep docs, error messages and skills agent-friendly (hypothesis)
Why it might work
Gauge's README lists outdated commands and confusing terminology as common failures, which docs work fixes.
What already exists
Docs platforms and technical-writing agencies already serve this market; the agent-testing loop is the differentiator.

Agent-shopping readiness checks for online stores

Who pays
Small e-commerce brands
How it would earn
One-time check plus fixes (hypothesis)
Why it might work
If shoppers send agents to compare and buy, a store whose pages confuse agents may lose sales.
What already exists
Not researched in depth. Commerce platforms may build this into their own products, which would make it a feature rather than a service.

Why we'd try "Fixed-fee agent experience audit for small developer-tool startups" first: Paying demand is visible at the top of the market, the tools are free, and a single failure found in a founder's own onboarding is a strong first pitch.

Build It

How you could build it

A five-day audit package: task list, agent runs, failure report, fixes, and a re-run.

First step: Install Gauge CLI, pick a developer tool you already use, and run one onboarding task with two coding agents. Write down the first place each agent got stuck.

Agents and tools

  • Gauge CLI
  • Microsoft Scope (optional)
  • Docker
  • Claude Code, Codex or Cursor subscriptions

Setup

  1. Write a reusable list of 10 onboarding tasks (install, auth, first call, common error).
  2. Run the tasks on two public tools to build sample reports.
  3. Create a one-page report template with before/after results.
  4. Set a fixed audit scope and turnaround.

What the agent does

  • Runs every task across several agents
  • Captures transcripts and failure points
  • Drafts candidate fixes to docs and error messages
  • Re-runs tasks after fixes

What you do

  • Chooses the tasks with the client
  • Judges which failures matter
  • Writes the final recommendations
  • Sells and presents the audit

Your first customer: A small developer-tool founder you contact with one real failure you found in their onboarding.

Test demand first: Send three founders one real failure each with a proposed fix. Success: at least one asks for the full audit.

Your starter instructions

Paste this into the AI agent you use (ChatGPT, Claude, Gemini, Grok Bot or another) as your first message. Copying it starts work in your own agent; it doesn't create or deploy a business by itself.

Get a launch plan

Running it

What stays human: Choosing realistic tasks with the client; judging which failures matter; writing the fix recommendations; presenting results; never changing the client's code without approval.

Startup cost (estimate)
About $0 plus agent subscriptions you may already have (Estimate: both tools are open source and run locally with your own agent plans, per their READMEs. Not measured)
Running cost
Agent usage per audit is unknown; Gauge's hosted product quotes about $3 to $4 per eval (according to its pricing page), which is a reference, not our cost.
How much the agent runs
People run it; AI helps with tasks.

Pricing your own version? Use the free startup-cost worksheet.

The evidence

What the source shows: The projects' own repositories and launch post. We read them; we haven't run a client audit.

What's ours: The fixed-fee audit is our application of the free tools.

Worth planning around

  • The free tools could let a startup do this in-house.
  • Agent behavior changes with each model version, so findings go stale quickly; date every report.

Sources

Sources reopened Oct 20, 2026. Spot a mistake? Tell us and we'll correct it in public.

Keep reading

The Daily Paycheck: one remarkable AI business at a time, and exactly how you could build around it. Free; unsubscribe anytime.

Get new AI business discoveries by email.