Morning triage

The Triage tab answers the first question of the day: what broke overnight, and why. Each morning it shows your base branch's failures grouped by cause, a category and a confidence for each group, the commits that probably caused it, and a next step.

The triage is not computed by razo.ar. It runs in your own CI with the open-source @razohq/triage engine, which reads the razo-steps.json artifacts of your recent runs and sends its report here. Grouping, classifying and naming suspects is deterministic: no AI model is involved, and no analysis credit is spent.

Set it up

You need a project on razo.ar with its ingest token in the repository secret RAZO_INGEST_TOKEN (see Dashboard & plans), and a test workflow that uploads test-results/ as an artifact on every run, green ones included, under a name that starts with razo-test-results-:

- uses: actions/upload-artifact@v4
  if: always()
  with:
    name: razo-test-results-${{ github.run_id }}-${{ github.run_attempt }}
    path: test-results/
    retention-days: 30

Install the engine:

npm i -D @razohq/triage

Add .github/triage/triage.config.yaml, replacing <owner>/<repo>:

baseBranch: main
source:
  plugin: razo-source
  config: { dataDir: .razo }
pull:
  repo: <owner>/<repo>
  token: "${GITHUB_TOKEN}"
  workflow: e2e.yml          # the workflow that runs your tests
  branch: main
code:
  plugin: github
  config: { repo: <owner>/<repo>, token: "${GITHUB_TOKEN}" }
notifiers:
  - plugin: http
    config: { url: "https://razo.ar/api/triage/reports", token: "${RAZO_INGEST_TOKEN}" }
store:
  plugin: json-file
  config: { path: .razo/triage-state.json }
# The decisions you take on the Triage tab, read back every morning.
decisions:
  url: "https://razo.ar/api/triage/decisions"
  token: "${RAZO_INGEST_TOKEN}"
# Optional: link failures to the GitHub issues that track them.
tracker:
  plugin: github
  config: { repo: <owner>/<repo>, token: "${GITHUB_TOKEN}" }

Then add .github/workflows/triage.yml. It runs after each nightly run of the test workflow on main, and keeps the triage's memory between mornings as an artifact:

name: triage

on:
  workflow_run:
    workflows: [e2e]          # the name of your test workflow
    types: [completed]
    branches: [main]
  workflow_dispatch:

concurrency:
  group: razo-triage
  cancel-in-progress: false

permissions:
  actions: read     # list runs, download their artifacts and the previous state
  contents: read    # compare commits and read their diffs
  issues: read      # only with the tracker: find the issues that track each failure

jobs:
  triage:
    # Only after the scheduled run, not after every push or pull request.
    if: github.event_name == 'workflow_dispatch' || github.event.workflow_run.event == 'schedule'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22, cache: npm }
      - run: npm ci
        env: { PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: '1' }
      - run: npx triage pull --config .github/triage/triage.config.yaml --since 14d
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          RAZO_INGEST_TOKEN: ${{ secrets.RAZO_INGEST_TOKEN }}
      - run: npx triage run --config .github/triage/triage.config.yaml --since 24h --lookback 14d
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          RAZO_INGEST_TOKEN: ${{ secrets.RAZO_INGEST_TOKEN }}
      - uses: actions/upload-artifact@v4
        with:
          name: razo-triage-state
          path: .razo/triage-state.json
          retention-days: 90

Run it once by hand (workflow_dispatch) to check the setup; the morning appears in the Triage tab. Until the first one arrives, the tab says so. The engine's README covers every config option, the token permissions in detail, and running it locally.

Reading a morning

The top of the tab lists the last 14 mornings, with the number of failure groups next to each date, and the selected morning's window: how many tests ran, how many failures, how many groups, and how many came back after being fixed.

Below it, one card per group of failures that share a cause (the same error, normalized), most urgent category first:

CategoryMeaning
regressionThe test passed in its last runs and fails on every attempt since one commit.
stale-testAn element the test uses is missing or hidden, and a recent commit changed it: the test may need updating, not the app.
environmentMany test files failed with network, timeout or 5xx errors within minutes of each other.
flakyThe test passed and failed on the same commit, or passed on retry.
unknownNot enough history to tell yet.

The confidence pill says how firmly the rules decided: high, medium or low. A stale-test is never high: telling an outdated test from a real regression takes a person. Each card also shows:

At risk

At the bottom, At risk lists tests that passed this morning on their first attempt but whose recent history on the base branch looks unstable. Each one shows a risk from 0 to 100% and why:

It is advice, not a verdict: nothing is proposed and nothing about the failure groups changes. It needs @razohq/triage 0.2.0 or later and at least 5 runs of history; the thresholds are under rules.forecast in the config, and maxTests: 0 turns it off.

Decisions

Each card has buttons to record what you decided. They change nothing in your repository or CI: the next morning's triage pull reads them and the engine applies them.

Ignored and flaky groups move to the Known section at the bottom, with the date of the decision. They stay there even if they go green and fail again, marked reopened, so an ignored failure does not come back by itself. To undo, use Stop ignoring or Not flaky: the group returns to the list the next time it fails. An acknowledge recorded before a group reopened no longer applies, and the card says so.

Issues

With the tracker section in the config, a regression or stale-test card offers Create issue in GitHub. It opens GitHub's new-issue form prefilled with the title, tests, range, suspects and evidence. You review it and submit it yourself: neither razo.ar nor the engine ever writes to GitHub.

The issue body carries a line razo-triage: <id> that identifies the group. Keep it: the next morning finds the issue through it, links the card to the issue, and stops proposing a new one. To link an issue that already exists, add that line to its body. A group that is already green again still offers the button, for when the fix was a temporary revert and you want to keep a record.

Next steps