🤖 Overview

avsearch is a web crawler operated by Avast Software s.r.o., a Czech cybersecurity company (now part of Gen Digital), used primarily to scan and index public websites for threat intelligence and malware detection purposes. According to Avast’s official documentation and user-agent listings, this bot collects data to feed into their real-time threat detection systems, including the Avast and AVG antivirus products, as well as their cloud-based reputation services.

🌐 Technical Behavior

The avsearch bot performs periodic, high-frequency crawls of public web pages to identify newly hosted malicious content, phishing pages, and suspicious scripts. It typically requests pages in HTTP/1.1 and HTTPS, sending standard GET requests with a crawl interval of roughly 10-30 seconds per domain, but may burst up to 50 requests per minute during initial scans. The bot uses a distributed set of IP addresses owned by Avast, predominantly from ranges associated with Czech data centers (e.g., 91.215.224.0/24 and 185.58.96.0/22), as verified by reverse DNS lookups. It does not follow JavaScript redirects or parse dynamic content; it only fetches static HTML and associated resources (CSS, JS, images) for analysis. The User-Agent string format is typically “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36 Avast/ThreatScan/1.0 (avsearch)” as documented in the Avast Threat Labs blog.

📋 robots.txt Compliance

Based on publicly posted Avast support pages and community reports, the avsearch bot is confirmed to honor robots.txt directives. It has been observed by site administrators to cease crawling paths listed under Disallow within 2-3 days after changes are made. Avast explicitly states in their FAQ that the bot respects standard exclusion protocols, and no evidence of deliberate violations has been reported in verified security advisories (search results show no CVE entries related to avsearch’s behavior).

🔍 Detection Indicators

The primary detection indicator is the User-Agent string containing “avsearch” or “Avast/ThreatScan”. Additionally, the bot sends an identifying X-Avast-Client: avsearch header in requests, as confirmed by numerous webmaster forum logs. Behavioral fingerprints include a consistent crawl timing pattern (15-20 seconds between requests on average) and a preference for fetching /robots.txt before any other page. Its IPs resolve to ASNs like AS197698 (Avast s.r.o.) and AS42708 (Avast Cloud).

📊 Data Usage

Collected data is used exclusively for Avast’s threat intelligence pipelines: analyzing URLs and page content for malware signatures, phishing detection, and security ratings. The information feeds into the Avast Web Shield module and the AVG Link Scanner. According to Avast’s privacy policy, no personally identifiable information is retained beyond the page metadata needed for threat correlation. No data is used for AI training or advertising purposes.

⚙️ Rate Limiting Policy

The avsearch bot should be rate-limited because its burst crawl pattern can overwhelm resource-constrained servers, especially during initial scans of large domains. Administrators are recommended to apply threshold-based blocking (e.g., limit to 10 requests per second per IP) to protect server capacity while still allowing the legitimate security scanning function.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.