Catching Breaking API Changes in CI/CD — Before Your Customers Do

API regression testing that a human has to remember to run is documentation, not protection. The whole point of a baseline-and-compare workflow is that it's cheap to run — which means it should run on every deploy, gated in the pipeline, with zero human involvement until something is actually wrong.

The pattern

Any CI-integrated regression platform needs exactly three API calls:

1. Trigger a snapshot when you bless a build — this becomes the new baseline.
2. Trigger a comparison after each deploy to a target environment.
3. Poll the execution until it completes, then decide: pass, warn, or fail the pipeline.

With APIpact the GitHub Actions step looks like this:

# .github/workflows/deploy.yml
- name: API regression gate
  run: |
    EXECUTION=$(curl -s -X POST \
      -H "Authorization: Bearer ${{ secrets.APIPACT_API_KEY }}" \
      https://console.apipact.dev/api/public/v1/content-regression/rest/$TEST_ID/comparison \
      | jq -r '.executionId')

    # Poll until finished
    while true; do
      STATUS=$(curl -s \
        -H "Authorization: Bearer ${{ secrets.APIPACT_API_KEY }}" \
        https://console.apipact.dev/api/public/v1/content-regression/executions/$EXECUTION)
      [ "$(echo $STATUS | jq -r '.status')" = "COMPLETED" ] && break
      sleep 10
    done

    # Fail the build on critical changes
    CRITICAL=$(echo $STATUS | jq -r '.summary.critical')
    if [ "$CRITICAL" -gt 0 ]; then
      echo "::error::$CRITICAL critical API regressions detected"
      exit 1
    fi

Decide what fails the build

The severity model is what keeps this gate from being disabled within a week. A useful default policy:

• CRITICAL (status changes, new errors, type changes, removed fields) → fail the pipeline.
• WARNING (value changes in significant fields) → pass, but post to the team channel and create a ticket.
• INFO (expected volatility: timestamps, trace IDs) → recorded, never surfaced.

The classification has to be trainable. The first week, you'll approve a batch of expected diffs; after that, the gate only trips on genuine surprises. A regression gate that cries wolf gets continue-on-error: true added to it, and then it protects nothing.

Don't forget pre-production — that's the point

The highest-value place for this gate is the deploy to staging, where breakage is free. If staging isn't reachable from the internet, that's what remote executors are for: the comparison job routes through an executor inside your network, and the pipeline logic stays identical.

Wiring this up takes about an afternoon. Keeping a storefront incident out of production pays for that afternoon roughly forever. Get in touch if you want help mapping your pipeline.