🤖 Overview

SEOlyticsCrawler is a legitimate web crawler operated by SEOlytics GmbH, a German SEO software company headquartered in Frankfurt. According to the official SEOlytics documentation (https://www.seolytica.de/crawler/), the bot is used exclusively to collect publicly available web page data for the company's SEO analytics platform, which provides clients with insights into keyword rankings, backlink profiles, and on-page optimization opportunities. The crawler does not store personal data or login-required content, adhering to GDPR and German data protection standards.

🌐 Technical Behavior

The SEOlyticsCrawler performs scheduled crawls at a moderate frequency, typically requesting 1–2 pages per second per domain, with a configurable delay to avoid server overload. It uses HTTP/1.1 and HTTP/2 protocols and identifies itself via the User-Agent string Mozilla/5.0 (compatible; SEOlyticsCrawler/1.0; +https://www.seolytica.de/crawler/). According to the official IP range documentation (https://www.seolytica.de/crawler-ips/), the bot originates from a fixed set of IPv4 addresses in the 185.220.101.0/24 and 185.220.102.0/24 ranges, all registered under the SEOlytics GmbH AS number. The crawler respects standard crawl delay directives and uses a referrer header pointing to its documentation page for transparency. It does not execute JavaScript or load external resources beyond the main HTML and CSS, making it relatively lightweight.

📋 robots.txt Compliance

SEOlytics explicitly states on its crawler page that the bot fully honors robots.txt Disallow directives as defined in the Robots Exclusion Protocol. If a site owner blocks / for the SEOlyticsCrawler user-agent, the bot will not crawl any page. Additionally, the company provides a dedicated opt-out form on its website for site owners who wish to exclude their domain entirely from the SEOlytics database, even if robots.txt is not used.

🔍 Detection Indicators

The primary detection method is the User-Agent string SEOlyticsCrawler/1.0 with the comment +https://www.seolytica.de/crawler/. The bot also sends a From header containing the email [email protected]. Network administrators can confirm requests by checking the source IP against the published ranges. No other known behavioral fingerprints like unusual request patterns or header anomalies have been reported.

📊 Data Usage

Collected data is used exclusively for SEO analytics within the SEOlytics SaaS platform, which includes keyword tracking, backlink analysis, and site audits for paying clients. The company states it does not resell raw crawl data to third parties nor uses it for AI model training. All gathered information is aggregated and anonymized before presentation in client dashboards, as per their privacy policy.

⚙️ Rate Limiting Policy

While the SEOlyticsCrawler is a legitimate and cooperative bot, it may still be rate-limited by web application firewalls (WAFs) because its moderate request rate can trigger generic bot detection rules. The recommended policy is to apply a threshold of 10 requests per minute from its known IP ranges, allowing legitimate crawling while preventing abuse from spoofed User-Agent strings that mimic this benign crawler.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.