SnipStudioGrowthBot is the crawler behind our website validator. It only visits a site when one of our users asks for a check of that site.
What it does
- Requests a limited number of HTML pages (typically 15–150), four at a time.
- Reads robots.txt, the XML sitemap and /llms.txt.
- Does not submit forms, log in, or store page content beyond titles, headings and a short text sample used for the report.
How to block it
Add this to your robots.txt or block the user agent in your firewall:
User-agent: SnipStudioGrowthBot Disallow: /