Questions · Discover

Should I check the sitemap my platform generates?

Yes, once, and then rarely. Generated sitemaps are commonly capped at a product count, split across child sitemaps where only the index is advertised, or excluded by a robots rule somebody added later for an unrelated reason. None of those failures produce a visible error: the sitemap still exists, still validates and still returns 200, it simply describes a fraction of your catalogue. Open it, count the URLs, and compare that against how many products you sell.

Why it happens

Link following is expensive for an agent and it is bounded. Every hop costs a fetch and a parse, and agents work under a budget, so deep or paginated catalogues get truncated. The products that fall outside the crawl are not ranked lower. They are absent.

A sitemap changes the shape of the problem from search to lookup. With one, an agent can confirm in a single request that a product exists and where it lives, which is the difference between answering a specific shopper question and offering the nearest thing on the homepage.

Run a free scan and find out which of these your store actually hits.