🤖 Overview

VMBot is a legitimate web crawler operated by VirusTotal (a Google subsidiary), designed to automatically fetch and analyze web content for the purpose of detecting malicious URLs, phishing sites, and malware distribution. The bot was first documented in VirusTotal’s official developer documentation and is a core component of the platform’s URL scanning service, which aggregates content from crawled pages to feed VirusTotal’s threat intelligence database.

🌐 Technical Behavior

VMBot typically crawls at a moderate rate, issuing GET requests to a single URL with a configurable delay (commonly 2–5 seconds between requests) to avoid overwhelming target servers. Its IP ranges are drawn from Google’s public IP blocks, as VirusTotal runs on Google Cloud infrastructure, but specific ranges are not published. The bot operates over HTTP/1.1 and HTTPS, and it fetches the full page content including HTML, JavaScript, and embedded resources to simulate real user interactions for thorough analysis. According to VirusTotal’s API documentation, the crawler may follow a limited number of redirects (typically up to 10) and respects Cache-Control headers to avoid redundant fetches. It does not execute JavaScript in a full browser environment but instead parses static HTML and optionally downloads referenced files for malware analysis.

📋 robots.txt Compliance

VirusTotal states in its official documentation that VMBot respects the robots.txt file, including the Disallow directive for paths that site owners wish to exclude from crawling. However, because the primary purpose is security scanning, the bot may still access URLs that are disallowed if they are submitted directly to VirusTotal for analysis; this behavior is documented on VirusTotal’s “Crawling and robots.txt” help page. Overall, for automated crawling without a direct submission, the bot follows standard robots.txt rules.

🔍 Detection Indicators

Identifying VMBot is straightforward via its User-Agent string: VMBot (case-sensitive) or VirusTotal-VMBot as documented in VirusTotal’s “User-Agent” section of their API reference. Additional indicators include requests originating from Google Cloud IP ranges (e.g., 34.64.0.0/10, 35.191.0.0/16), a lack of a Referrer header in most cases, and a consistent Accept header of text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8. The bot does not send cookies or a custom X-Forwarded-For header.

📊 Data Usage

The content fetched by VMBot is used exclusively for VirusTotal’s threat detection services: the page content and extracted resources are analyzed by multiple antivirus engines and machine learning models to classify URLs as benign or malicious. This data is also used to populate VirusTotal’s public and private URL reputation databases, and aggregated statistics are made available to security researchers and enterprise customers through the VirusTotal API (e.g., the /urls endpoint). No personal user data is retained beyond the scope of scan results.

⚙️ Rate Limiting Policy

VMBot is rate-limited because it aggressively scans submitted URLs and may walk links to find hidden threats, disproportionately consuming server resources on sites with many subpages. Threshold-based blocking is acceptable to protect application performance; administrators may set a limit of, for example, 10 requests per second per IP before returning a 429 status, as the bot will retry with backoff due to its configurable delay logic. This policy aligns with common security crawling practices and does not impede legitimate security scanning.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.