scrybot
ScryBot is a legitimate crawler operated by Scry, the company behind the Scry AI search product, documented at https://scry.ai/bot. It discovers publicly accessible pages that feed Scry's search index and retrieval-augmented answer pipeline; it is not user-triggered and does not execute JavaScript or submit forms. Scry's public bot page is the authoritative source for its current user-agent and opt-out instructions.
ScryBot sends HTTP GET requests over TLS, follows redirects, and fetches HTML plus limited linked resources for indexing. Scry publishes no fixed CIDR block; official docs direct validation by User-Agent, forward-confirmed reverse DNS, and Scry-owned cloud infrastructure rather than hard-coded IP ranges. It identifies itself on every request and uses per-host politeness, adaptive concurrency, and backoff on 429 or 503 responses. Requests come from distributed cloud egress addresses, so source IPs change as capacity scales.
Scry's documentation states ScryBot honors robots.txt Disallow rules and will not intentionally fetch disallowed paths. Operators can exclude it with a targeted User-agent: ScryBot group or wildcard rule; Scry recommends robots.txt for preferences and firewall rules for immediate enforcement. The policy cites RFC 9309 for parsing and caching. Scry documents that robots.txt rules are fetched and cached per host and rechecked periodically.
The main fingerprint is the literal ScryBot token in User-Agent, commonly ScryBot/1.0 (+https://scry.ai/bot). Scry relies on no secret header, so combine UA matching with verified DNS/IP ownership and request-rate patterns because UA strings are spoofable. No CVE or security advisory is associated with ScryBot in NVD as of this writing; it is a crawler, not an exploitation tool.
Pages fetched by ScryBot build and refresh Scry's search index and support AI-generated answers with citations. Scry's documentation describes indexing publicly available content for search retrieval; it is not an analytics beacon or advertising tracker. Site owners can review Scry's bot page for opt-out and data-use details.
ScryBot is rate-limited because unconstrained crawling can consume origin bandwidth, CPU, and database capacity; threshold blocking with 429 responses and temporary IP/UA blocks protects availability while allowing indexing to resume after backoff. The rationale is predictable crawl traffic and site-specific enforcement without permanently blocking Scry's crawler. Operators should return 429 with Retry-After when possible so ScryBot can back off gracefully.
Similar Threats
Free Bot Analysis
Find out exactly how much of your traffic is automated — and which bots are draining your bandwidth and skewing your analytics.
Run Free Bot Scan →No credit card required · Results in minutes
ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.
Stay up to date with the latest from Boteraser.
We use cookies to improve your experience on our site. By using our site, you consent to cookies.
Manage your cookie preferences below:
Essential cookies enable basic functions and are necessary for the proper function of the website.
CloudFlare provides web performance and security solutions, enhancing site speed and protecting against threats.
Service URL: developers.cloudflare.com (opens in a new window)
These cookies are needed for adding comments on this website.
These cookies are used for managing login functionality on this website.
Statistics cookies collect information anonymously. This information helps us understand how visitors use our website.
Google Analytics is a powerful tool that tracks and analyzes website traffic for informed marketing decisions.
Service URL: policies.google.com (opens in a new window)
You can find more information in our Cookie Policy and Privacy Policy.