🤖 Overview

Toplistbot is a web crawler operated by Toplist.cz, a Czech web analytics and directory service founded in 1996. Its primary purpose is to collect website traffic data, page titles, and metadata for the Toplist.cz ranking and statistics platform, which provides site owners with free visitor counters and aggregated analytics.

🌐 Technical Behavior

The bot typically crawls sites that have voluntarily placed a Toplist.cz counter or are listed in the Toplist directory. It requests pages sequentially with a moderate crawl rate of around 1–2 requests per second per domain, though this may vary based on server response times. It uses standard HTTP/1.1 and HTTP/2 protocols and resolves to IP addresses primarily from the Czech Republic (ASN 43516, Toplist s.r.o.) and occasionally from other European locations. The crawler does not follow query parameters or dynamic URLs by default, focusing only on static HTML pages where counter scripts are embedded.

📋 robots.txt Compliance

According to the official documentation on toplist.cz, Toplistbot fully respects Disallow directives in robots.txt. It also checks for the X-Robots-Tag HTTP header and can be excluded via meta tags. The bot’s behavior is documented on the Toplist.cz help pages, which explicitly state that site owners can block it by adding User-agent: Toplistbot Disallow: / to their robots.txt.

🔍 Detection Indicators

The primary User-Agent string is Toplistbot (exact case‑sensitive) with a version suffix such as Toplistbot/2.0. It also sets a custom header From: [email protected] for contact purposes. The bot’s IP ranges are published on the Toplist.cz website, typically covering the subnet 185.95.93.0/24. Behavioral fingerprints include a low request frequency and the absence of JavaScript execution—only HTML content is fetched.

📊 Data Usage

Data collected by Toplistbot is used exclusively for the Toplist.cz analytics service: creating public and private website rankings, generating visitor statistics, and providing site owners with detailed traffic reports. No personal or sensitive data is stored; only publicly accessible page titles, response times, and referrer patterns are aggregated. The bot does not perform AI training or sell data to third parties.

⚙️ Rate Limiting Policy

Toplistbot is rate‑limited because its steady crawl pattern can still produce noticeable load on high‑traffic sites, especially when scraping large directories. Most web applications apply a threshold block (e.g., more than 100 requests per minute from its IP range) to prevent resource exhaustion while still allowing legitimate statistical data collection.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.