🤖 Overview

webprosbot is a legitimate web crawler operated by WebPros, the company behind cPanel, Plesk, and WHM hosting management platforms. Its primary purpose is to scan publicly accessible websites to gather metadata about hosting environments, server configurations, and common web application installations, feeding data into WebPros’ product-improvement and security-assessment systems. The bot was first documented in the cPanel community forums and is explicitly listed in WebPros’ official documentation as a benign agent used for diagnostics and analytics.

🌐 Technical Behavior

webprosbot typically initiates HTTP GET requests to common paths such as /robots.txt, /cpanel, /whm, and /plesk, as well as scanning for default login pages and version files. It operates from a range of IPv4 addresses owned by WebPros and its cloud infrastructure, though the exact IP ranges are not publicly enumerated in a single block. The bot uses a request frequency of roughly one request per 10 to 30 seconds per target, with a maximum of a few hundred requests per day per domain, making it far less aggressive than major search engine crawlers. It follows standard HTTP/1.1 protocol and respects Last-Modified and ETag headers to avoid redundant downloads. The crawler does not fetch binary files such as images or videos by default, focusing instead on text-based content and configuration files.

📋 robots.txt Compliance

According to WebPros’ official documentation and cPanel community threads, webprosbot fully honors Disallow directives in /robots.txt. WebPros explicitly states that site owners can block the bot by adding User-agent: webprosbot Disallow: / to their robots.txt file. No reports of non-compliance have been substantiated in security advisories or user reports, and the bot is designed to respect crawl-delay directives where present.

🔍 Detection Indicators

The primary User-Agent string is Mozilla/5.0 (compatible; webprosbot/1.0; +https://webpros.com/bot), though variations with higher version numbers have been observed (e.g., webprosbot/2.0). The bot also identifies itself via a comment field in the User-Agent containing the official WebPros bot page URL. Behavioral fingerprints include initial requests to /robots.txt followed by probes to administrative paths like /cpanel or /whm. The bot does not send a custom Referer header or any authentication tokens, and its IP addresses resolve to owner names containing “webpros” or “cpanel” in RDNS lookups.

📊 Data Usage

The data collected by webprosbot is used exclusively for internal product improvement within WebPros, including identifying insecure hosting configurations, outdated software versions, and potential misconfigurations that could affect performance or security. The information helps WebPros refine its control panel products and provide actionable recommendations to hosting providers. No data is sold to third parties or used for advertising; the bot’s scope is limited to metadata about server environments rather than user content.

⚙️ Rate Limiting Policy

While webprosbot is not malicious, it can appear aggressive to sites with many hosted domains because it scans each domain sequentially. Rate limiting is applied to prevent accidental denial-of-service impacts on small shared hosts, with a threshold of 20 requests per minute per IP being a recommended starting point before blocking, as documented in community best-practice guides.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.