🤖 Overview

WebWatch is a legitimate website monitoring crawler operated by Catchpoint Systems Inc., first documented in 2018 and widely used for performance and availability monitoring of web applications. Its primary purpose is to simulate real user visits from multiple global vantage points to measure page load times, DNS resolution, and error rates, feeding data into Catchpoint’s synthetic monitoring platform. Unlike search engine bots, WebWatch does not index content; it collects timestamped performance metrics for enterprise clients.

🌐 Technical Behavior

WebWatch crawls using HTTP/1.1 and HTTP/2 protocols, with requests mimicking a typical Chrome 80 browser user-agent. It fetches only top-level pages and critical resources (CSS, JavaScript, images) to measure full page load performance, but it does not follow internal links beyond the initial URL. According to Catchpoint’s official IP whitelist documentation (docs.catchpoint.com/docs/ip-ranges), the crawler originates from a fixed set of about 120 IPv4 addresses distributed across AWS regions (us-east-1, eu-west-1, ap-southeast-1) and Catchpoint’s own backbone. Requests are sent at intervals configurable by the customer, typically once every 5 to 60 minutes per monitored URL, with a maximum of 2 concurrent connections per target. The bot sends a Cache-Control: no-cache header and a custom X-Catchpoint-Test: 1 header to differentiate itself.

📋 robots.txt Compliance

Based on Catchpoint’s official robots.txt best practices guide (docs.catchpoint.com/docs/robots-txt), WebWatch strictly honors Disallow directives in the site’s robots.txt file. The crawler reads robots.txt at the start of each test session and skips any URL path marked as disallowed. However, it does not cache robots.txt across sessions, so changes take effect within the next test cycle. Administrative endpoints and dynamic pages can be safely blocked.

🔍 Detection Indicators

The primary User-Agent string is WebWatch/1.0 (compatible; Catchpoint Systems; +https://www.catchpoint.com/bot). A secondary variant may appear as Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36 Catchpoint/1.0. Behavioral fingerprints include requesting only the root path plus a single resource (e.g., / and /style.css) without subsequent clicks, and the absence of Accept-Encoding: gzip in some older tests. The X-Forwarded-For header often contains the actual monitoring node IP.

📊 Data Usage

Collected performance data—such as Time to First Byte (TTFB), DOM content loaded time, and HTTP status codes—is aggregated into Catchpoint dashboards and alert systems. It is not used for AI training, search indexing, or advertisement targeting. The raw metrics are stored encrypted for up to 90 days per Catchpoint’s data retention policy (see catchpoint.com/legal/data-retention). Enterprise clients can export logs for internal analysis.

⚙️ Rate Limiting Policy

WebWatch is rate-limited because its frequent, automated requests from distributed IPs can mimic low-level HTTP floods. Threshold-based blocking (e.g., >10 requests/sec from a single IP) is justified to protect server resources while allowing legitimate monitoring; Catchpoint recommends whitelisting its IP ranges rather than blocking the bot entirely.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.