evlog map in CI

A score you look at once is a nice afternoon. A score in CI is what stops the next handler from shipping dark.

The gate

--min-score <n> prints an explicit verdict and exits 1 when the project is below the threshold:

evlog map --min-score 80

 GATE  score 76 is below --min-score 90 — exit code 1
fix what is listed under FIX FIRST to pass · evlog.dev/cli/ci

 GATE  score 76 meets --min-score 70 — exit code 0

The full report is printed either way, so a failed job tells you what to fix without a second run.

The threshold has to be a whole number between 0 and 100. Anything else — a typo, a stray unit, an unexpanded shell variable — stops the command rather than being read as "no gate", because a gate that quietly disables itself reports success for a bar nobody checked.

Which turns the score into something a pull request can move. One run says the app is short of the bar and names the three entry points responsible, the next one says it is over:

The ratchet: --baseline

--min-score asks "is this app good enough". --baseline asks the other question: did this pull request make it worse.

evlog map --baseline

It compares the fresh scan against the evlog.map.json you committed, per entry point and per check:

 BASELINE  score 56 → 44 (-12) vs evlog.map.json

REGRESSED
✗ POST /api/checkout — useLogger no longer passes
   server/api/checkout.post.ts · evlog.dev/learn/wide-events
✗ POST /api/checkout — log.audit no longer passes
   server/api/checkout.post.ts · evlog.dev/use-cases/audit/overview

NEW AND DARK
⚠ GET /api/reports — added with no instrumentation

2 regressions and a 12 point drop — exit code 1 · evlog.dev/cli/ci
evlog.map.json was not rewritten — fix the regression, or re-run without --baseline to accept it

The unit is the requirement, not the total. A refactor that instruments one route and breaks another can leave the score untouched, and a gate watching only the number would call that a no-op.

What counts as a regression

Silencing a check costs the same as breaking it, on purpose: a disable comment is the cheapest way to turn a gate green without writing any instrumentation, and a ratchet that let it through would be measuring the comments rather than the code.

New dark routes are reported rather than gated. On an app that is not green yet, failing every pull request that adds an endpoint is how a team learns to turn the job off — --min-score is where a bar for new work belongs. Run both when you want both.

The map file is the ratchet

--baseline means you commit evlog.map.json instead of ignoring it:

evlog map            # updates the file
git add evlog.map.json

A run that reports a regression deliberately leaves the file untouched. Overwriting it there would move the ratchet down to the worse state, and the same command run a second time would report no regression and exit 0. Accepting a drop is therefore explicit: re-run without --baseline, and commit the new map with the reason in your message.

      - uses: actions/checkout@v5
      - run: pnpm install --frozen-lockfile
      - run: pnpm evlog map --baseline

The file also churns on generatedAt every run. That is the cost of tracking it, and it buys a reviewable diff of exactly which entry points changed class.

Comparing against a branch instead

Pass a git:<ref> to read the committed copy through git rather than the working tree — useful when the pull request touches the map file itself:

evlog map --baseline git:origin/main
evlog map --baseline ../base-map.json   # or any path

With a bare --baseline and no map on disk, the CLI falls back to git:HEAD on its own, so the answer stays "what did the last commit say" rather than "what did I say a minute ago".

Exit codes

GitHub Actions

name: Observability

on: pull_request

jobs:
  map:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v5
        with:
          node-version: 22
      - run: npx @evlog/cli map --min-score 80 --no-write

--no-write keeps the job from producing an evlog.map.json nobody will read.

Pin the version

The CLI is young: rules are still being refined and new ones will be added, so a release can change a verdict on code nobody touched. On a gated job that shows up as a pull request failing for reasons its author cannot see in the diff.

Install the CLI as a dev dependency and let your lockfile hold it still:

pnpm add -D @evlog/cli

      - run: pnpm install --frozen-lockfile
      - run: pnpm evlog map --min-score 80 --no-write

Then a score change is always something you did, and upgrading the CLI is its own pull request — where a moved score is the point rather than a surprise.

Ratchet, don't cliff

Setting the threshold to 90 on an app that scores 41 fails every pull request and teaches the team to ignore the job. Set it to today's score, then raise it as you fix things:

# what are we at right now?
evlog map --json --no-write | jq '.map.score'

Each pull request that raises the score raises the floor with it. The report already tells you what the next step is worth: ▲ 76 → 86 by fixing the 3 above.

The JSON contract

--json writes the whole map to stdout. The report goes to stderr, so the two never mix.

evlog map --json --no-write > map.json

{
  "schemaVersion": 2,
  "environment": "production",
  "map": {
    "version": 1,
    "generatedAt": "2026-07-25T18:42:10.114Z",
    "framework": "nuxt",
    "projectName": "evlog-playground",
    "score": 76,
    "routes": []
  },
  "summary": { "instrumented": 19, "partial": 2, "dark": 8, "exempt": 0, "suppressedChecks": 0 },
  "mapPath": "/path/to/app/evlog.map.json"
}

mapPath is null under --no-write. Each entry in map.routes looks like this:

{
  "framework": "nuxt",
  "kind": "api",
  "method": "POST",
  "path": "/api/auth/login",
  "file": "server/api/auth/login.post.ts",
  "handler": { "line": 1, "column": 0 },
  "id": "337325358269",
  "checks": {
    "wide-event": { "status": "pass" },
    "context": { "status": "pass" },
    "structured-errors": { "status": "n/a" },
    "error-handling": { "status": "n/a" },
    "audit": {
      "status": "fail",
      "message": "has logger + context but no log.audit() — sensitive route needs audit trail",
      "evidence": { "file": "server/api/auth/login.post.ts", "line": 1 }
    }
  },
  "suggestions": {},
  "sensitivity": { "level": "high", "reasons": ["auth: path says \"auth\""] },
  "score": 75
}

checks holds requirements, suggestions holds opportunities. They are separate keys precisely so a consumer — or a CI script — can never mistake a suggestion for a failure. Rule ids are stable: the registry and the published id union are checked against each other at build time, so a release cannot silently change what you receive.

A check somebody disabled with a comment is n/a with "suppressed": true, and its evidence points at the comment rather than at the handler:

"wide-event": {
  "status": "n/a",
  "suppressed": true,
  "message": "disabled at line 1 — liveness probe, deliberately silent",
  "evidence": { "file": "server/api/health.get.ts", "line": 1 }
}

summary.suppressedChecks is the project total. It is the number to watch alongside the score: the gate only measures what the rules were allowed to look at.

Recipes

# the score, for a badge or a comment
evlog map --json --no-write | jq '.map.score'

# every entry point with no event at all
evlog map --json --no-write \
  | jq -r '.map.routes[] | select(.checks["wide-event"].status == "fail") | .file'

# every failure as file:line — message
evlog map --json --no-write \
  | jq -r '.map.routes[].checks | to_entries[] | select(.value.status == "fail")
           | "\(.value.evidence.file):\(.value.evidence.line) — \(.value.message)"'

# fail a script when anything is dark
test "$(evlog map --json --no-write | jq '.summary.dark')" -eq 0

# how much of the score is disabled checks
evlog map --json --no-write | jq '.summary.suppressedChecks'

The map file

Unless you pass --no-write, every run writes evlog.map.json to the project root. It holds the same data as --json and is a build artifact, so ignore it:

evlog.map.json

Track it instead if you want the diff in review — a pull request then shows exactly which entry points changed class, which is a useful thing to argue about. Just expect the file to churn on every run, since generatedAt changes each time. Committing it is also what --baseline compares against.

Monorepos

evlog map scans one app at a time. Gate each one:

jobs:
  map:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        app: [apps/web, apps/admin]
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v5
        with:
          node-version: 22
      - run: npx @evlog/cli map --cwd ${{ matrix.app }} --min-score 80 --no-write

Different apps can carry different thresholds, which is usually what you want: the app that takes payments should be held higher than the marketing site.

Next

• Rules — what a failing check means and how to fix it
• Scoring — how the number you are gating on is calculated