🤖 Overview

SeobilityBot is a legitimate web crawler operated by the German SEO company Seobility GmbH, headquartered in Cologne. Its primary purpose is to analyze websites for technical SEO issues, page structure, meta tags, link quality, and overall search engine friendliness. The data collected feeds directly into the Seobility SEO Checker tool (available at seobility.net/en), which provides website owners with detailed audit reports and optimization recommendations. The bot was first documented publicly around 2016 and has since become a standard tool for SEO professionals, alongside similar agents from Ahrefs and Semrush. According to Seobility’s official documentation, the crawler is designed only to visit publicly accessible pages and does not attempt to bypass login forms or scrape private content.

🌐 Technical Behavior

SeobilityBot performs a single-pass crawl of a submitted URL, typically following up to 200 internal links per scan (configurable by the user). The default crawl depth is 3 levels, with a maximum of 5,000 pages per audit for paid plans. Requests are made over HTTPS using the HTTP/1.1 protocol, with a standard Accept header and User-Agent string indicating the bot’s identity. Seobility publishes its IP ranges in two IPv4 blocks: 5.45.106.0/24 and 5.45.107.0/24 (verified via their official bot page). The crawler sends requests at a rate of approximately one request per second (1–2 seconds per page), with bursts of up to 5 concurrent connections during the initial discovery phase. It does not render JavaScript or execute client-side code, relying purely on static HTML analysis. The bot respects Cache-Control headers and will wait for a reasonable response; timeouts are set to 30 seconds per request.

📋 robots.txt Compliance

Based on Seobility’s own documentation and independent testing, SeobilityBot fully honors robots.txt directives. The official bot page states: “Our crawler observes the Disallow rules specified in your robots.txt file. If you wish to block our crawler entirely, add User-agent: SeobilityBot followed by Disallow: / to your robots.txt.” Multiple webmaster forums confirm that the bot stops crawling immediately upon encountering a Disallow rule and does not bypass it via alternate user-agent strings. It also respects the Crawl-Delay directive if present, with a minimum delay of 5 seconds per page.

🔍 Detection Indicators

The primary identification string is: Mozilla/5.0 (compatible; SeobilityBot; https://www.seobility.net/en/bot.html). In server logs, it may also appear as SeobilityBot/1.0 or simply SeobilityBot. The bot includes a custom From header bearing the same URL: From: https://www.seobility.net/en/bot.html. Reverse DNS lookups for the IP ranges mentioned above resolve to hostnames like crawl-5-45-106-x.seobility.net. A behavioral fingerprint is the consistent 1–2 second request interval and the absence of any Referer header.

📊 Data Usage

Collected data is used exclusively for SEO analysis and reporting within the Seobility platform. The tool generates reports covering page titles, meta descriptions, headings (H1–H6), image alt attributes, internal/external link metrics, page load speed, and mobile-friendliness. No data is sold to third parties or used for AI training. Seobility’s privacy policy (dated 2024) explicitly states that crawled content is stored temporarily and deleted after 30 days unless the user saves a report.

⚙️ Rate Limiting Policy

SeobilityBot is rate-limited because its single-threaded crawl pattern can still generate hundreds of requests in a short period when scanning large websites (e.g., e-commerce stores with many products). Threshold-based blocking is recommended to prevent server strain, setting a limit of 50 requests per minute (adjusted based on server capacity). The bot officially supports Crawl-Delay and will comply with a 5‑second minimum delay, making it one of the more cooperative SEO crawlers.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.