Questions · Discover

The pages open fine in my browser. Why did they not answer for you?

A browser carries a session, runs scripts and follows redirects with cookies in hand. An agent does none of that. The common causes are a route that needs a session, a redirect that a bot is excluded from, a rate limit that trips on rapid requests, and a feed still naming URLs the storefront has stopped serving. The report lists the exact URL and the exact status for each one, so you can repeat the request yourself.

Why it happens

An agent reaches a product the same way a link does. It takes the URL your catalogue states and requests it once, with no cookies, no session and no JavaScript. If that request does not return the product page, the agent has no second route to the same product, because the catalogue gave it one address and that address is the one it used.

The reason this survives unnoticed is that the pages usually work in a browser. A route that only resolves for a logged in session, a redirect chain that a bot is excluded from, a rate limit that trips on the third request in a second, a feed still naming last season’s URLs: each of those leaves the storefront perfectly usable for a person and empty for an agent.

It also hides inside a good report. Every check we make about price, stock, variants and identifiers reads a product page, so when no page can be read those checks find no fault to report. A store in this state can look clean while nothing about its products has been read at all, which is why this finding exists and why the stages behind it are left unscored rather than scored well.

This comes up at the Discover stage. The full treatment, including how to reproduce it and what to change, is on Product pages did not answer when the agent asked for them.

Terms in this answer

Related questions

Run a free scan and find out which of these your store actually hits.