robots.txt
A robots.txt is a file at the root of a site stating which automated clients may request which paths. It is a convention that well behaved operators follow, not an enforcement mechanism.
The distinction that matters for a store is that search crawling, training data collection and agentic shopping are three separate uses with three separate consequences. Most rules about AI were written thinking about the second and quietly decided the third.
Because it is a convention rather than enforcement, a disallow mostly stops the clients you might have wanted: those are the ones that read the file. Enforcement happens at the edge, and when the two disagree the edge wins.
Why it matters to a store
A rule written years ago to keep content out of a training set is, today, frequently the reason a store is absent from assistant driven shopping. Nothing announces that, and the rule still says exactly what it said.