How do I check MCP server quality in a CI pipeline?

Run `vouch check` (from @vouch-tools/cli) as a step in your pipeline, or use the AivaStack/vouchtools/packages/cli GitHub Action. It reads your project's .mcp.json, looks up each declared server's current Vouch behaviour score, and exits non-zero if any tool scores below your threshold.

Will the check fail my build for a tool Vouch hasn't measured yet?

No. Only an actual measured score below the threshold fails the build. A server Vouch can't match, matches ambiguously, or hasn't measured (insufficient_data / not_yet_tested) is reported and skipped — never a reason to fail.

What is the default threshold?

70. Override it with --threshold on the CLI or the threshold input on the Action. The score is Vouch's 0–100 measured behaviour score.

Does the CI check send my code or arguments anywhere?

No. `vouch check` only reads your MCP config file locally and makes read-only lookups against the public Vouch API. It sends no code, no arguments, and no credentials. Opt-in real-usage telemetry is a separate feature you have to enable explicitly.

CI check

Gate your CI on your MCP tools

Your project declares MCP servers in an .mcp.json. vouch check looks up each one’s current measured behaviour score and fails the build if a tool scores below your threshold — the npm audit model, for MCP dependencies. It runs on every PR; almost nobody has to open this site.

Quickstart

From any project directory that has an MCP config:

npx @vouch-tools/cli check

It finds .mcp.json (or mcp.json) — the same mcpServers format Claude Code, Claude Desktop, Cursor, and most MCP clients use — resolves each declared server against Vouch, and prints a line per tool:

✓ io.github.acme/files/read_file — score 94.2 >= threshold 70
✗ io.github.acme/files/search_files — score 41.0 < threshold 70
· com.example/weather/forecast — not yet tested
? internal-tools — not found in Vouch corpus

1 passed, 1 failed, 1 unmeasured, 1 not found (threshold 70)

Exit code is non-zero here because one measured tool is below 70. The unmeasured and unmatched lines are reported but never fail the build — see below.

GitHub Actions

A composite action wraps the same vouch check — no separate implementation to drift from the CLI:

# .github/workflows/vouch.yml
name: Vouch check
on: [pull_request]

jobs:
  vouch:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: AivaStack/vouchtools/packages/cli@main
        with:
          threshold: "70"          # optional, defaults to 70
          # config: .mcp.json      # optional, auto-detected
          # working-directory: .   # optional

It runs npx @vouch-tools/cli@latest check under the hood — deliberately unpinned, because a CI gate should reflect the current score, not a frozen CLI version. Exit codes pass straight through: the job fails exactly when vouch check would.

Any other CI

There’s no GitHub dependency in the check itself. Anywhere that can run npx — GitLab CI, CircleCI, a pre-commit hook, a Makefile target — this is the whole step:

npx --yes @vouch-tools/cli@latest check --threshold 70

What fails the build — and what deliberately doesn't

Only an actual measured score below your threshold fails the build. Everything else is reported and skipped:

Result	Effect on the build
✗ measured, below threshold	Fails (exit 1).
✓ measured, at or above threshold	Passes.
· not yet tested / insufficient data	Reported, does not fail. A CI check must never fail on data Vouch simply hasn’t collected yet.
? not found / ambiguous match	Reported, does not fail. Never guessed.

The one exception is exit code 3 — no config found at all. A workflow that explicitly added this check but points at nothing has a real setup problem, so that does fail.

Configuration

CLI flag	Action input	Default	Meaning
--threshold	threshold	70	Minimum behaviour score (0–100) a measured tool must meet to pass.
--config	config	auto-detected	Path to the MCP config. Defaults to .mcp.json, then mcp.json, in the working directory.
--api	api	https://vouch.tools	Base URL of the Vouch public API.
--json	—	off	Machine-readable report instead of the human listing. (CLI only.)

Exit code	Meaning
0	Every measured tool met the threshold. Unmeasured, ambiguous, and not-found servers do not count against this.
1	At least one tool scored below the threshold.
2	Bad invocation — unknown or missing subcommand.
3	No MCP config found, or it declared no servers. Nothing was checked.

What it sends

vouch check reads your MCP config file locally and makes read-only lookups against the public API. It sends no code, no arguments, and no credentials. Opt-in real-usage telemetry — reporting anonymised success/failure from MCP calls your own code makes — is a separate feature you enable explicitly; it is not part of the CI check.

Run it before you add a server, too

The same check works outside CI. Before adding an unfamiliar MCP server to a project, point vouch check at a config containing just that server — or look it up directly:

npx @vouch-tools/cli check --threshold 80

Or from an MCP client, call vouch_check on the Vouch MCP server — same data, in the place an agent is already working.

Back to the search, the methodology, or the operator page. The CLI is @vouch-tools/cli on npm.