---
title: "Release gates: failing a build when AI agents stall"
source: "https://ottom.io/docs/ci"
summary: "An Ottom release gate fails a CI build when a release makes your store harder for AI agents to buy from. Your pipeline starts a scan of your store or its staging origin through the Ottom API, waits for it, and reads one field: gate.result. This page gives the rules, the two API calls and a recipe for GitHub Actions, GitLab CI and CircleCI."
---

# Release gates: failing a build when AI agents stall

An Ottom release gate fails a CI build when a release makes your store harder for AI agents to buy from. Your pipeline starts a scan of your store or its staging origin through the Ottom API, waits for it, and reads one field: gate.result. This page gives the rules, the two API calls and a recipe for GitHub Actions, GitLab CI and CircleCI.

## How the gate decides

- Each run is compared with the last completed scan of your production store. A run on your staging origin is compared with production too, so a release is judged against what shoppers have now.
- For the overall score and for each stage you choose the largest drop a release may cause, in points. The gate fails when the score falls by more than that.
- A drop only counts once it is larger than the noise floor: twice the measured variation of the AI agents that scored the run, and never less than 5 points. A threshold of 0 therefore means any real drop, not any wobble.
- You can also fail the gate when a stage stalls, or when any stage stalls. A stall needs no earlier scan to compare with.
- When our scoring changes version, scores either side of the change are not compared, so drops are skipped for that run and stalls are still checked.

## What gate.result can be

- `pass`: Every threshold that could be checked held. The recipes pass the build.
- `fail`: At least one threshold tripped. gate.tripped says which. The recipes fail the build.
- `pending`: The scan is still running, its buy attempt included. Poll again.
- `skip`: The gate did not run: no thresholds apply to this store, or none could be checked this run. gate.reason says which. The recipes print it and pass.
- `error`: The scan could not finish, so there was nothing to judge. The recipes print it and pass. Change that line if you would rather fail.

## Setting it up

- Mint a CI token. On your workspace page, an owner or admin mints a CI token for the store. It is shown once. A token reaches one store and that store’s staging origin, nothing else.
- Store it as a secret. Save the token in your CI system as a secret named OTTOM_TOKEN, and the address to scan as OTTOM_STORE_URL: your store, or its staging origin.
- Add the script and the job. Commit the script below as ci/ottom-gate.sh and add the job for your CI system after the step that deploys.

## The two API calls

Both take the token as `Authorization: Bearer`. The base address is https://ottom.io/api/ci/runs.

- `POST /api/ci/runs`: Starts a scan. The body is JSON with url, the address to scan. Answers 202 with runId. Each scan counts against the store’s daily limit, and a store at its limit answers 429 with code rate_limited.
- `GET /api/ci/runs/{runId}`: Reads the scan: its status, score, stages, findings your plan can see, cost, and gate. gate.result is the field a pipeline acts on, gate.tripped lists the thresholds that failed, and gate.receiptUrl links the receipt for a ticket.

## The script

Saved as ci/ottom-gate.sh. Every recipe below runs it.

```
#!/bin/sh
# Ottom release gate. Needs curl, jq, OTTOM_TOKEN and OTTOM_STORE_URL.
set -eu
API="https://ottom.io/api/ci/runs"
AUTH="Authorization: Bearer $OTTOM_TOKEN"

START=$(curl -sS --fail-with-body -X POST "$API" -H "$AUTH" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --arg url "$OTTOM_STORE_URL" '{url: $url}')") || { echo "$START"; exit 1; }
RUN_ID=$(echo "$START" | jq -r '.runId')
echo "Ottom scan $RUN_ID started"

RESULT=pending
i=0
while [ "$RESULT" = pending ] && [ "$i" -lt 60 ]; do
  sleep 30
  i=$((i + 1))
  REPORT=$(curl -sS --fail-with-body "$API/$RUN_ID" -H "$AUTH") || { echo "$REPORT"; exit 1; }
  RESULT=$(echo "$REPORT" | jq -r '.gate.result // "pending"')
done

echo "$REPORT" | jq -r '"Gate: \(.gate.result). \(.gate.reason)", "Receipt: \(.gate.receiptUrl)"'
case "$RESULT" in
  fail) exit 1 ;;
  pending) echo "No gate result after 30 minutes."; exit 1 ;;
  *) exit 0 ;;
esac
```

## GitHub Actions

In `.github/workflows/ottom-gate.yml`. Runs when your staging deploy workflow finishes. Add OTTOM_TOKEN under Settings, Secrets and variables, Actions. Change the workflow name to yours.

```
name: Ottom release gate
on:
  workflow_run:
    workflows: ["Deploy staging"]
    types: [completed]
jobs:
  gate:
    if: github.event.workflow_run.conclusion == 'success'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Ottom release gate
        env:
          OTTOM_TOKEN: ${{ secrets.OTTOM_TOKEN }}
          OTTOM_STORE_URL: https://staging.example.com
        run: sh ci/ottom-gate.sh
```

## GitLab CI

In `.gitlab-ci.yml`. A job after your deploy job. Add OTTOM_TOKEN as a masked variable under Settings, CI/CD, Variables.

```
ottom_gate:
  stage: test
  needs: [deploy_staging]
  image: alpine:3.20
  variables:
    OTTOM_STORE_URL: https://staging.example.com
  before_script:
    - apk add --no-cache curl jq
  script:
    - sh ci/ottom-gate.sh
```

## CircleCI

In `.circleci/config.yml`. A job that requires your deploy job. Add OTTOM_TOKEN as a project environment variable or in a context. The cimg/base image already has curl and jq.

```
version: 2.1
jobs:
  ottom-gate:
    docker:
      - image: cimg/base:stable
    environment:
      OTTOM_STORE_URL: https://staging.example.com
    steps:
      - checkout
      - run:
          name: Ottom release gate
          command: sh ci/ottom-gate.sh
workflows:
  release:
    jobs:
      - deploy-staging
      - ottom-gate:
          requires: [deploy-staging]
```

## Questions

### Does a release gate cost extra scans?

Each gate run is one scan of the store or its staging origin, and it counts against that store’s daily limit like any other. A store at its limit answers 429 and the script fails the step, so a busy pipeline should gate once per release rather than once per commit.

### Why did a drop of a few points not fail the gate?

AI agents do not score a store identically every time. A drop has to be larger than the noise floor, at least 5 points, before the gate treats it as a change your release made. How the floor is measured is on the scoring page.

### What is the difference between gate and verdict in the response?

verdict compares the run with the previous run of the same store and fails on new findings at a severity you pick. gate compares it with your production store and fails only on the thresholds you set. The recipes read gate. Both are in every response.

### Can the gate scan a store that is not public yet?

Yes, if it is a staging origin set up for the store. A token for the store can scan that origin, and the run is judged against the production store.