🤖 Overview

SeobilityBot is the automated web crawler operated by Seobility GmbH, a German SEO software company headquartered in Cologne, Germany. The bot is the core engine behind the Seobility Website Analyzer, a commercial tool that performs technical SEO audits by crawling and analyzing website structure, meta tags, page speed, broken links, and on-page optimization factors. According to Seobility's official documentation at seobility.net and their blog posts, the crawler is designed to mimic search engine behavior to help webmasters identify issues that affect search rankings.

🌐 Technical Behavior

SeobilityBot initiates crawls from IP ranges belonging to Amazon Web Services (AWS) and DigitalOcean, as identified in publicly listed User-Agent logs. The crawler sends requests at a configurable rate, but by default it respects a delay of at least 1 second between requests from the same IP, as documented in their crawler behavior guide on seobility.net/en/crawler/. It uses HTTP/1.1 and HTTP/2 protocols and supports both IPv4 and IPv6 addresses. The bot can optionally execute JavaScript when the user enables the "JavaScript Rendering" feature, in which case it employs a headless Chromium browser. SeobilityBot follows redirect chains up to a depth of 10 hops and can crawl up to 500 URLs per site per scan by default, though paying users can increase this limit.

📋 robots.txt Compliance

SeobilityBot fully honors the standard robots.txt exclusion protocol, as stated in their official FAQ at seobility.net/en/faq/. If a webmaster disallows the SeobilityBot User-Agent in robots.txt, the crawler will not access those paths. Additionally, the bot checks for X-Robots-Tag HTTP headers and meta robots directives on each page, stopping further crawl of pages tagged with noindex or nofollow.

🔍 Detection Indicators

The primary User-Agent string is SeobilityBot (e.g., SeobilityBot/1.0 (+https://seobility.net/en/bot)). The bot also identifies itself with the comment Email: [email protected] in the User-Agent string. Behavioral fingerprints include a consistent crawl rate of ~3–5 requests per second from a single IP and a non-standard header Seobility-Crawl-ID that contains a unique scan identifier. Web servers can log these headers to differentiate SeobilityBot from other crawlers.

📊 Data Usage

All data collected by SeobilityBot is used exclusively for the Seobility Website Analyzer tool, which generates SEO audit reports, page-level scores, and actionable recommendations for website owners. The data is not sold to third parties and is retained only as long as necessary for the scan results, as per Seobility's privacy policy. The company does not use the data for AI training or for building any public index; it is purely for the individual client's analysis.

⚙️ Rate Limiting Policy

Although SeobilityBot is legitimate and rate-limited by its own settings, web application owners may choose to apply threshold-based blocking to prevent excessive resource consumption during high-traffic periods. The recommended rate limit is 10 requests per second per IP, as the bot's default crawling speed rarely exceeds this threshold; blocking above that ensures minimal disruption while still allowing the valuable SEO audit service to function.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.