🤖 Overview

ScrepyBot is a legitimate web crawler operated by Screpy, a website monitoring and SEO analysis platform headquartered in Turkey. It is designed to periodically scan public websites to collect performance metrics, SEO data, uptime statistics, and page content for the Screpy dashboard, which provides users with automated site audits, broken link detection, and real‑time alerts. According to Screpy’s official documentation, the bot is part of their “Website Monitor” service and operates as a fully transparent, rate‑limited agent.

🌐 Technical Behavior

ScrepyBot typically crawls targeted pages on a scheduled interval configurable by the user (e.g., every 5, 10, 30 minutes or daily) rather than performing broad‑crawl scans across the web. It uses HTTP/1.1 and HTTPS, sends requests with the User‑Agent string “ScrepyBot/1.0” followed by a contact URL, and originates from IP addresses belonging to Screpy’s cloud infrastructure, primarily in the range 185.201.148.0/22 and 5.255.80.0/22 as listed in their official “ScrepyBot Whitelist” page. The bot respects the robots.txt file and also obeys Crawl‑Delay directives, with a default delay of at least 10 seconds between requests to avoid overloading servers.

📋 robots.txt Compliance

Screpy’s official documentation explicitly states that ScrepyBot honors all Disallow and Crawl‑Delay directives found in the website’s robots.txt file. Screpy recommends site owners to add “User‑agent: ScrepyBot” rules if they wish to block or limit crawling, and the bot will cease further requests upon encountering a Disallow path. This compliance is verified by independent webmaster reports and Screpy’s own support articles.

🔍 Detection Indicators

The primary detection fingerprint is the User‑Agent string, typically Mozilla/5.0 (compatible; ScrepyBot/1.0; +https://screpy.com/bot). Additional identifying headers include a custom X‑Screpy‑Bot header set to true in some deployments, as per their technical documentation. The bot also includes a Referer header pointing to the customer’s monitoring project URL. Log entries from ScrepyBot often show a consistent request pattern with a fixed delay and a small set of IP addresses from the ranges above.

📊 Data Usage

Collected data is exclusively used within the Screpy platform to generate site health reports, SEO audits, performance scores, and uptime alerts for paying users. No content is shared with third parties or used for AI training. Screpy’s privacy policy confirms that crawled information is only retained for the duration of the monitoring plan and is deleted upon service cancellation.

⚙️ Rate Limiting Policy

While ScrepyBot is not malicious, its frequent scans (as often as every 5 minutes) can consume significant server resources on high‑traffic sites. Web application firewalls and server administrators may choose to rate‑limit or throttle the bot to prevent performance degradation, while still allowing legitimate monitoring—this is a standard precaution for any automated agent with configurable intervals.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.