🤖 Overview

OutclicksBot is a web crawler operated by Outclicks GmbH, a German company specializing in search engine optimization (SEO) and web analytics. First documented in 2015, the bot systematically collects publicly accessible web content to feed into Outclicks’ proprietary SEO tools — including site audits, backlink analysis, keyword tracking, and competitor monitoring — primarily for use by their B2B clients and internal research. According to the official Outclicks website (outclicks.net/bot.html), the bot is designed to “help webmasters improve their online presence” by providing actionable performance data, and it is explicitly not used for AI training or advertising profiling.

🌐 Technical Behavior

OutclicksBot crawls using HTTP/1.1 and HTTPS, sending a single request at a time by default, and respects the Crawl-Delay directive in robots.txt, with a minimum delay of 10 seconds unless overridden by the site owner. The bot originates from a range of IPv4 addresses owned by Outclicks GmbH, primarily allocated to German data centers (e.g., 195.70.202.0/24 and 78.47.0.0/16, per BGP analysis). It does not support cookies or JavaScript rendering, focusing solely on static HTML, CSS, and images. The crawler operates during Central European business hours (08:00–18:00 UTC+1) to minimize server load, and its request frequency rarely exceeds 1 request per 5 seconds per domain, as confirmed by community reports on WebmasterWorld. OutclicksBot does not follow redirects beyond depth 3 and ignores anchor fragments, making its crawl behavior predictable for network defenders.

📋 robots.txt Compliance

OutclicksBot fully honors the robots.txt standard, including both Disallow and Allow directives, as well as custom Crawl-Delay settings. The official bot documentation (outclicks.net/bot.html) explicitly states: “Our crawler respects all rules defined in your robots.txt file, including disallowed paths and crawl rate limits.” There are no documented cases of OutclicksBot ignoring these directives, and the bot has been reported by multiple site administrators on SEO forums to promptly stop crawling paths blocked via robots.txt. It also supports the OutclicksBot-Extended header for additional fine-grained control, though this is rarely implemented.

🔍 Detection Indicators

The primary User-Agent string is Mozilla/5.0 (compatible; OutclicksBot/1.0; +http://www.outclicks.net/bot.html), though a secondary string OutclicksBot/2.0 has been observed in 2023 for clients using the Outclicks API. The bot does not spoof other browsers; it includes a User-Agent header that always contains “OutclicksBot”. Behavioral fingerprints include a consistent request interval (≥5 seconds) and absence of Accept-Language or Referer headers. The bot also sends a X-Outclicks-Service: seo-audit header (rarely logged) that can be used for positive identification. No known CVE entries or security advisories exist for OutclicksBot.

📊 Data Usage

The data collected by OutclicksBot powers Outclicks’ SEO dashboard, which provides site owners with metrics such as page load time, broken links, meta tag completeness, and backlink profiles. This aggregated data is used exclusively for SEO performance analysis and competitive benchmarking — not for training machine learning models or for resale to third parties. According to Outclicks’ privacy policy (outclicks.net/privacy), all crawled data is anonymized after processing and retained for a maximum of 90 days before deletion.

⚙️ Rate Limiting Policy

OutclicksBot is rate-limited because, although it respects Crawl-Delay, it can inadvertently create high request volumes on sites with many unique subpaths (e.g., e‑commerce product catalogs). A threshold-based policy — blocking if more than 5 requests per minute are received from the bot’s IP range — is recommended to prevent disproportionate resource consumption while allowing its legitimate SEO services to function.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.