Stall library · Discover
Product pages did not answer when the agent asked for them
A store whose catalogue lists products and whose product pages do not answer is a store an AI shopping agent can see and cannot read. The feed or the sitemap names a URL, the agent fetches it, and the answer is a 404, a redirect that ends somewhere else, or no answer at all. Nothing about the product is knowable after that: not its price, not its stock, not whether the size a shopper asked for exists.
Why the agent stops
An agent reaches a product the same way a link does. It takes the URL your catalogue states and requests it once, with no cookies, no session and no JavaScript. If that request does not return the product page, the agent has no second route to the same product, because the catalogue gave it one address and that address is the one it used.
The reason this survives unnoticed is that the pages usually work in a browser. A route that only resolves for a logged in session, a redirect chain that a bot is excluded from, a rate limit that trips on the third request in a second, a feed still naming last season’s URLs: each of those leaves the storefront perfectly usable for a person and empty for an agent.
It also hides inside a good report. Every check we make about price, stock, variants and identifiers reads a product page, so when no page can be read those checks find no fault to report. A store in this state can look clean while nothing about its products has been read at all, which is why this finding exists and why the stages behind it are left unscored rather than scored well.
How often we see this
Not published yet. Ottom does not have a benchmark dataset large enough to state how often this stall occurs, so this space is empty rather than filled with an estimate. The figure will appear here when it exists, alongside the methodology behind it.
See it for yourself
- Open your product feed or your sitemap and copy three product URLs exactly as they are written there.
- Fetch each one in a private window with JavaScript turned off, or with curl, and check the status is 200.
- Follow any redirect to the end and confirm it lands on that product rather than on a collection or the home page.
- Request all three within a second or two, the way an agent would, and check the third still answers.
What to change
- Make every URL your catalogue states answer 200 with the product on it, for a request carrying no cookies and running no scripts.
- Regenerate the feed and the sitemap whenever a product URL changes, so neither keeps naming a route the storefront no longer serves.
- Where a redirect is unavoidable, point it at the product itself rather than at a collection page or the home page.
- Check the rate limits and firewall rules in front of the storefront, and allow the published Ottom user agent through them.
What Ottom looks at
Ottom samples product URLs from your catalogue, your sitemap or your home page links, in that order, and fetches each one. This finding is raised when pages were sampled and not one of them could be read, and the status each URL returned is listed on the report. The evaluate and compare stages are then left out of the score rather than scored from nothing.
Terms used here
Questions
The pages open fine in my browser. Why did they not answer for you?
A browser carries a session, runs scripts and follows redirects with cookies in hand. An agent does none of that. The common causes are a route that needs a session, a redirect that a bot is excluded from, a rate limit that trips on rapid requests, and a feed still naming URLs the storefront has stopped serving. The report lists the exact URL and the exact status for each one, so you can repeat the request yourself.
Why is my score lower than the findings suggest it should be?
It is not lower. Two stages are missing from it. Everything we can say about price, stock, variants and identifiers comes from reading a product page, so with no page read we leave the evaluate and compare stages out of the score entirely. Scoring them well would be telling you nothing is wrong with data nobody has looked at.
Is this the same as bot protection blocking you?
No, and the report says which it is. A challenge page identifies itself, and we raise bot protection blocks the agent instead. This finding is what is left when there is no challenge in the way and the pages simply do not answer.