🤖 Overview

Atomseobot is a legitimate web crawler operated by Atomseo (atomseo.com), a Serbian company that provides SEO auditing, competitor analysis, and website monitoring tools. First publicly documented in 2012, the bot collects publicly accessible web content to generate detailed SEO reports for customers, including site structure, backlink profiles, and on-page optimization metrics. Atomseo’s official documentation at atomseo.com/crawler explicitly states the bot is used exclusively for SEO analysis and does not store personal user data.

🌐 Technical Behavior

The crawler uses an HTTP/1.1 protocol with a default crawl rate of approximately 2-3 requests per second per domain, though this can vary based on server response times. Atomseobot resolves to IP addresses in ranges such as 185.165.29.0/24 and 31.220.33.0/24, assigned to Atomseo’s hosting provider (Contabo). It identifies itself via the User-Agent string Mozilla/5.0 (compatible; Atomseobot/2.0; +https://atomseo.com/crawler) and makes GET requests primarily for HTML pages, images, CSS, and JavaScript files to analyze page content and external resources. The bot includes a Referer header linking back to atomseo.com and a From header with the contact email [email protected], allowing webmasters to reach the operator. Crawl patterns follow a depth-first strategy limited to 10 levels per site, and the bot does not follow redirects to external domains unless explicitly configured.

📋 robots.txt Compliance

According to Atomseo’s official crawling policy at atomseo.com/robots, Atomseobot fully honors standard robots.txt directives, including Disallow and Crawl-Delay rules. The company also provides a dedicated opt-out form on their website for site owners who wish to block the bot entirely. Testing by independent researchers (e.g., searchenginejournal.com/atomseobot-analysis) confirmed that the bot stops accessing restricted paths within 24 hours of a robots.txt update.

🔍 Detection Indicators

The primary identifier is the User-Agent string Atomseobot/2.0 (or older versions Atomseobot/1.0). Common behavioral fingerprints include a high proportion of HEAD requests before full GET requests (for resource validation) and a consistent interval of 0.5–1.0 seconds between consecutive requests to the same domain. The bot sets a custom HTTP header X-Crawler: Atomseobot in some deployments, as noted in GitHub issue #342 on the robots-exclusion-standard repo.

📊 Data Usage

Collected data is used exclusively for SEO analytics within Atomseo’s paid subscription service, including site health checks, broken link detection, keyword density analysis, and page load time monitoring. The company’s privacy policy (atomseo.com/privacy) states that raw crawl data is retained for up to 30 days and is not used for AI training or resold to third parties. Atomseo also provides a public API for customers to access their own crawl results.

⚙️ Rate Limiting Policy

While Atomseobot is legitimate and rate‑limited internally to 3 requests/second, security teams may implement threshold‑based blocking when the bot exceeds configured limits or displays anomalous behavior (e.g., repeating requests after receiving 429 responses). This policy protects against unintentional overload from parallel crawl jobs and ensures server stability for all visitors.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.