CI setup

razo's two CLIs cover two independent CI integrations: uploading runs to a hosted dashboard, and posting an AI failure analysis directly on a pull request with your own Anthropic key. Neither depends on the other — add one, both, or neither. On Pro, there's also a third, zero-config path: connect a repo once in the dashboard and razo posts the analysis it already computed straight onto the PR, with no extra CI step and no API key — covered below.

Recipe: upload runs to the dashboard

Create a project at razo.ar/dashboard and copy its ingest token (shown once — see Dashboard & plans). Add it as a repository secret named RAZO_INGEST_TOKEN, and make sure @razohq/razo-analyzer is in your devDependencies — it provides the razo-upload CLI:

npm install -D @razohq/razo-analyzer

Upload with if: always() so both passing and failing runs reach the dashboard — a run with zero failures is still useful history, and a run that fails outright still needs to show up:

name: e2e

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npx playwright install --with-deps

      - name: Run Playwright tests
        run: npx playwright test

      - name: Upload razo artifacts
        if: always()
        run: npx razo-upload --url https://razo.ar/api/ingest
        env:
          RAZO_INGEST_TOKEN: ${{ secrets.RAZO_INGEST_TOKEN }}

Self-hosting? Point --url at your own deployment instead of razo.ar. --url can also come from a RAZO_INGEST_URL repository variable instead of hardcoding it in the step. razo-upload reads GITHUB_* environment variables automatically, so branch, commit, PR number and the CI run link are attached to the run with no extra configuration.

The PR number only exists on pull_request events. razo-upload reads it out of GITHUB_REF, which is refs/pull/<n>/merge only when the workflow was triggered by a pull request. A run triggered by push, by schedule, or by the Run workflow button (workflow_dispatch) carries no PR number, and everything downstream of it is skipped silently: no PR comment, and no diff for the analysis to reason about. The run still uploads and still gets analyzed — it just gets analyzed blind. If you are re-running a suite to see a PR comment, re-run the original pull_request run from the Actions tab rather than dispatching a new one; a re-run replays the original event payload.

Recipe: analyze failures on pull requests (bring your own key)

This recipe doesn't touch the dashboard at all — it runs razo-analyze --pr-comment directly, using your own ANTHROPIC_API_KEY, and posts the result as a PR comment. It only needs to run when tests actually fail:

name: e2e

on: pull_request

permissions:
  pull-requests: write

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npx playwright install --with-deps

      - name: Run Playwright tests
        run: npx playwright test

      - name: AI failure analysis
        if: failure()
        run: npx razo-analyze --pr-comment
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

--pr-comment needs GITHUB_TOKEN explicitly mapped into the step's environment (it isn't injected by default) and permissions: pull-requests: write on the job, or the comment post is rejected. Re-runs update the same comment instead of stacking new ones.

razo-analyzer also ships a composite GitHub Action that wraps the same command, if you'd rather not spell out the npx call:

- name: Run Playwright tests
  run: npx playwright test

- name: AI failure analysis
  if: failure()
  uses: razohq/razo/packages/razo-analyzer@main
  with:
    anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
    results-dir: test-results

It defaults pr-comment to true and still requires the same permissions block.

Recipe: hosted PR comments (Pro, no API key)

If you're on Pro and already uploading runs to the dashboard, razo can post the analysis it already computed on ingest straight onto the pull request — no second analysis, no Anthropic key, and no extra CI step. One run is analyzed once and serves both the dashboard and the PR comment.

Setup is one-time and lives in the dashboard, not your workflow:

Open your project's Settings → GitHub PR comments → Connect GitHub.
Install the razo GitHub App on the repository (or organization) you want. The App needs Pull requests: read — see Reading the change under test below for why.
Back in Settings, pick the repository this project posts to.

That's the whole configuration. Your existing razo-upload step (from the first recipe) already attaches the PR number on pull_request runs, so once the project is connected, every PR run with failures gets a comment. Re-runs update the same comment instead of stacking new ones. Passing-only PR runs post nothing, since there's no failure to analyze.

The comment is deliberately short — a classification, a one-line verdict, the failing tests, and a link:

### razo — 2 failed · 2 passed

**Intentional change** — PR intentionally hides the Place order button and
removes the Mouse row; tests still assert the old behaviour.

- `placing the order confirms it` — tests/checkout.spec.ts
- `the cart lists both items` — tests/checkout.spec.ts

[See the full analysis on razo →](https://razo.ar/dashboard/…)

The long-form root cause and the per-test diagnoses live on the linked run, not in the pull request. Nobody reads six paragraphs inside a code review.

Reading the change under test

A test failure on its own cannot tell you whether the application broke or whether it changed on purpose and the test went stale — the failure looks identical either way. So when a run carries a PR number, razo also reads that pull request: the files it changed, and the description its author wrote.

Two things come out of that. The diff gets intersected with the controls the failing tests actually used, so the analysis can say this PR changed the very button this test clicks rather than guessing. And the description supplies intent, which no diff contains — a deliberate redesign and a shipped bug produce identical hunks.

That's what lets the verdict lead with a classification:

Classification	Meaning
`Product regression`	The application is broken; the test is right
`Intentional change`	The application changed on purpose; the tests assert the old behaviour
`Outdated test`	The expectation or the locator no longer matches a working application
`Missing precondition`	The test never performed a step it depended on
`Flake`	Timing or environment; the same code would pass on a retry
`Undetermined`	The evidence does not settle it

Undetermined is a real answer, not a failure mode. Separating a regression from an intentional change requires the diff, so if razo cannot read the pull request it says so instead of guessing — a confident wrong cause costs a reviewer more time than no cause at all.

If your verdicts keep coming back Undetermined, the most likely reason is that the GitHub App lacks Pull requests: read. Older installations predate that permission, and GitHub requires each installation to approve a permission change before it takes effect: check Settings → Applications → razo → Configure on the repository or organization. The second most likely reason is that the run had no PR number at all — see the note in the first recipe.

Reading the pull request costs nothing extra: it happens inside the analysis you're already paying for, and the diff is capped so a large PR can't inflate it.

Because the comment reuses the analysis razo already ran for the dashboard, it costs nothing beyond your normal ingest — there's no separate quota for it. The Pro line is posting the comment, not reading the pull request: connect a repo on Free and your trial analyses are still read against the diff and the description, they just land in the dashboard instead of on the PR. See Subscription plans.

Hosted vs. bring-your-own-key: use hosted PR comments if you're on Pro and want zero CI config with no API key to manage — the dashboard does the posting. Use the bring-your-own-key recipe above if you're on the free/OSS path: it runs entirely in your CI with your own Anthropic key and never touches the dashboard or its quota. You wouldn't normally run both against the same PR — they'd post two separate comments.

Combining both

The two recipes read the same test-results/ directory and don't interfere — add both steps to one job: razo-upload with if: always() keeps the dashboard's run history and AI diagnosis complete, razo-analyze --pr-comment (or the composite Action) with if: failure() gives reviewers an immediate comment on the PR without waiting to open the dashboard. Since the PR-comment recipe uses your own API key, it isn't subject to the dashboard's hosted analysis quota — see Dashboard & plans.

Next steps

CLI reference — every flag, environment variable and exit code for both binaries.
Dashboard & plans — tokens, run history and hosted quotas.
Subscription plans — what Pro unlocks, including hosted PR comments.