Documentation

How an Ottom score is calculated

An Ottom score is a weighted average of four stages an agent must get through to buy from a store: discover, evaluate, compare and buy. This page gives the weights, what each finding costs, how the buy stage is walked, and the two rules that keep noise out of alerts: a noise floor on score changes, and two signals before a Critical.

The four stages

• Discover

  Whether an agent is let in at all, by robots.txt and bot protection, and what your store publishes for machines to read: sitemaps, product feeds, structured data and llms.txt. Scored from the findings it raises. The stall library lists 8 findings for this stage.

• Evaluate

  Shopper requests are matched against your product data alone: price, stock, variants, identifiers and policies, as an agent can read them without running the page. Rules decide what they can, and a language model judges only the matches rules cannot settle. Scored from the findings it raises. The stall library lists 6 findings for this stage.

• Compare

  Checks the facts a comparison rests on: that your feed and your storefront agree on price, that shipping cost can be known before checkout, and that the catalogue has categories to compare within. Scored from the findings it raises. The stall library lists 3 findings for this stage.

• Buy

  A browser walks the purchase path on your live store and records how far it gets. The walk is a script rather than a language model, because the failures that end most purchases are mechanical, and every run records which driver walked it. It stops at the payment step and never pays. The score comes from how far it got, and its findings say where and why it stopped. The stall library lists 8 findings for this stage.

What a finding costs

A data stage starts at 100 and loses points for each finding. At 80 or above it is PASS, from 50 to 79 it is SLOW, and below 50 it is STALLED. It never goes below zero.

How the buy stage is scored

The buy stage is PASS, at 100, only when the walk reaches the payment step. Anything short of that is STALLED, scored by how far it got, up to 45: a store that reached checkout scores above one that stopped at the size picker, and neither passes.

The walk takes these steps, in order:

1. Open the product
2. Choose an option
3. Add to cart
4. Open the cart
5. Start checkout
6. Reach the payment step

The score itself

The score is the weighted average of the stages a scan measured. When a scan could not measure a stage, the report says which stages it covers rather than scoring the missing one as zero or as a pass.

The noise floor

Two scans of an unchanged store do not always score the same, because stores and networks vary from minute to minute. So an alert reports a score change only when it is larger than 5 points and larger than twice the run to run variation measured for that kind of scan. Smaller changes still go into your score history. They are not announced as a regression or an improvement.

Two signals before a Critical

A Critical finding is alerted on, and counted as open in the weekly digest, only when two signals agree: two agent journeys, or one journey backed by a retrieval check. A retrieval check alone never confirms a Critical, because it cannot see the purchase.

Versions

Scores are calculated by scoring version score-v2 and findings are named by stall-v2 of the stall taxonomy, which has 25 codes. Every run records the version that scored it, and a change to the weights, the costs or a severity is released as a new version rather than made silently. Every code is published at /taxonomy.json.

Questions

How scores are combined across stores is on the benchmark methodology. How often each plan scans is on the plans page.