Learn / For your team
Adding agent checks to your CI pipeline
Agent readiness decays through ordinary changes, so the only version that survives is one asserted automatically. A useful check fetches a small set of product URLs without executing scripts and asserts that price, availability and variant options are present, which catches most regressions in seconds and fails against the change that caused it.
Agent readiness decays through entirely ordinary changes: a theme update, an app install, a component refactor. A check somebody remembers to run does not survive that. A check that runs on every deploy does.
What to assert
Keep it small enough to run on every commit, which means fetching markup rather than driving a browser.
- Fetch a fixed set of product URLs: a bestseller, one with many variants, one out of stock.
- Assert price and currency are present in the served markup.
- Assert availability is present, and that the out of stock one says so.
- Assert variant options are present with labels.
- Assert the add to cart control is a form with an action.
That runs in seconds and catches the majority of what regresses.
Fail the build
A warning that does not fail is ignored within a fortnight, and it is then worse than nothing because it creates the appearance of coverage. If you are not willing to fail on it, do not assert it.
Choose the URLs deliberately
The out of stock product is the one that matters most and the one most likely to be omitted, because it is awkward to keep a permanently out of stock item. Use a fixture product if you need to: the point is to assert the negative case, which is where availability data is usually wrong.
See how to expose stock availability to agents.
The slower checks
Full journey checks, cold session, narrow viewport, reaching the payment step, are too slow for every commit and belong on a schedule and on checkout changes. Splitting fast assertions from slow journeys is what keeps the fast half fast enough to survive.
Questions
What should the check assert?
Presence of price, availability and variant options in the served markup on a fixed set of URLs, plus that the add to cart control is a form. Those four cover most of what breaks.
Should it fail the build?
Yes, if it is asserting things you have decided you need. A check that only warns gets ignored within a fortnight, which is worse than not having it because it creates the appearance of coverage.
What about the checkout half?
Harder to assert cheaply and worth a periodic run rather than a per-commit one. Keep the fast markup assertions on every deploy and the full journey on a schedule.