🤖 Overview

tswebbot is a web crawler operated by Trend Micro, a cybersecurity firm headquartered in Tokyo. Its sole purpose is to support Trend Micro's Web Reputation Service (WRS), which continuously evaluates website safety to protect users of Trend Micro products. According to official documentation at trendmicro.com/tswebbot, the bot has been active for over a decade and is explicitly non‑malicious, helping detect phishing, malware, and drive‑by downloads.

🌐 Technical Behavior

The crawler sends HTTP/1.1 GET requests from IPs in Trend Micro's own ASN (AS23861) and cloud providers. Crawl intervals range from seconds to minutes, and the bot limits its own rate to avoid overload. It uses User‑Agent tswebbot and includes a From header with [email protected]. It fetches HTML, JavaScript, and images, analyzing JavaScript for obfuscated code. Trend Micro's policy caps requests at 10 per second per domain, but actual rates are throttled dynamically based on server response.

📋 robots.txt Compliance

Trend Micro confirms that tswebbot honors both robots.txt and X‑Robots‑Tag headers. Site owners can block it with User‑agent: tswebbot and Disallow: /. Real‑world tests and community reports verify compliance, and Trend Micro provides a contact form for concerns.

🔍 Detection Indicators

The primary User‑Agent is tswebbot (case‑sensitive), with variants like tswebbot/1.0. Reverse DNS resolves to crawler-*.trendmicro.com. The From header always includes an @trendmicro.com email. The bot lacks typical browser headers (e.g., Accept‑Language, Referer) and does not set cookies.

📊 Data Usage

Collected data—URL metadata, page content, and JavaScript behavior—is aggregated into Trend Micro's Smart Protection Network to assign safety ratings (trusted, suspicious, dangerous). This rating feeds real‑time blocking in Trend Micro products. No PII is intentionally collected, and data is not used for AI training or advertising.

⚙️ Rate Limiting Policy

Although tswebbot is legitimate, its deep scanning of JavaScript can be resource‑intensive. A threshold‑based rate limit (e.g., >100 requests per minute) is recommended to protect server performance while still allowing the bot to complete its security assessments.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.