Quality gates

An ordered pipeline, cheapest checks first, so an expensive stage is only reached once the earlier ones have passed. A profile may supply the command for a stage; the project may override it; review and human approval cannot be disabled.

This console runs nothing. The harness can run these gates from the command line, with ax run --execute, after checking that the acting role holds the capability to do so. Reporting is the default, and every gate shown here is unrun.

The four outcomes

This distinction is where honest tooling is usually lost. A gate nobody could run and a gate nobody did run have not passed, and a pipeline that gates nothing does not report success.

passedThe command ran and succeeded.
failedThe command ran and did not succeed.
unavailableNo command exists for an applicable gate. Nobody could run it.
unrunA command exists, but it was not run.

The pipeline

01BuildRuns a declared commandcommands.build
The project compiles or packages. Nothing later is meaningful if this fails.
02LintRuns a declared commandcommands.lint
Style and correctness rules the project has chosen to enforce.
03TypecheckRuns a declared commandcommands.typecheck
Static types agree across the change.
04Unit testsRuns a declared commandcommands.test
Fast tests that do not need external services.
05Integration testsRuns a declared commandcommands.integration_test
Tests that exercise real collaborators, and may need a database or network.
06SecurityRuns a declared commandcommands.security
Automated dependency and configuration checks. Not a substitute for security review.
07Independent reviewRequires a person
A reviewer who did not write the change assesses it. Cannot be disabled.
08Agent evaluationAssesses agent behaviour
Assesses how the agent worked, which application tests cannot measure.
09Human approvalRequires a person
A person accepts the change. Cannot be disabled.

Which gates each profile can supply

A profile supplies only the commands a project of that kind reliably has. Everything else must be declared by the project, or the gate stays unavailable rather than quietly passing.

java-spring
Supplies build, unit tests. The project must declare lint, typecheck, integration tests, security.
nextjs-react
Supplies build, security. The project must declare lint, typecheck, unit tests, integration tests.
fullstack
Supplies nothing. The project must declare build, lint, typecheck, unit tests, integration tests, security.

pipeline / default

{
  "schemaVersion": 1,
  "id": "default",
  "description": "Ordered quality pipeline. Cheap and broad checks run first so that an expensive stage is only reached once the earlier ones have passed.",
  "stages": [
    {
      "id": "build",
      "title": "Build",
      "kind": "executable",
      "quality": "build",
      "command": "build",
      "description": "The project compiles or packages. Nothing later is meaningful if this fails."
    },
    {
      "id": "lint",
      "title": "Lint",
      "kind": "executable",
      "quality": "lint",
      "command": "lint",
      "description": "Style and correctness rules the project has chosen to enforce."
    },
    {
      "id": "typecheck",
      "title": "Typecheck",
      "kind": "executable",
      "quality": "typecheck",
      "command": "typecheck",
      "description": "Static types agree across the change."
    },
    {
      "id": "unit_tests",
      "title": "Unit tests",
      "kind": "executable",
      "quality": "tests",
      "command": "test",
      "description": "Fast tests that do not need external services."
    },
    {
      "id": "integration_tests",
      "title": "Integration tests",
      "kind": "executable",
      "quality": "integration_tests",
      "command": "integration_test",
      "description": "Tests that exercise real collaborators, and may need a database or network."
    },
    {
      "id": "security",
      "title": "Security",
      "kind": "executable",
      "quality": "security",
      "command": "security",
      "description": "Automated dependency and configuration checks. Not a substitute for security review."
    },
    {
      "id": "review",
      "title": "Independent review",
      "kind": "human",
      "quality": "review",
      "description": "A reviewer who did not write the change assesses it. Cannot be disabled."
    },
    {
      "id": "eval",
      "title": "Agent evaluation",
      "kind": "eval",
      "quality": "eval",
      "description": "Assesses how the agent worked, which application tests cannot measure."
    },
    {
      "id": "human_approval",
      "title": "Human approval",
      "kind": "human",
      "quality": "human_approval",
      "description": "A person accepts the change. Cannot be disabled."
    }
  ]
}