🤖 Overview

SeoStar is a legitimate web crawling agent operated by the SEO analytics firm SeoStar Ltd., which provides competitive SEO auditing, backlink profiling, and content gap analysis for enterprise clients. The bot systematically scans public websites to gather ranking signals, on-page metadata, and link structures, feeding into the company’s proprietary SeoStar Dashboard used by marketing teams for search-optimization strategies. According to official documentation on seostar.io/robots, the bot is designed exclusively for SEO research and does not store personally identifiable information.

🌐 Technical Behavior

The bot employs a depth-first crawl strategy with a maximum of 25 concurrent connections per domain and a per-request delay of 2 seconds, as stated in the project’s technical specification. Requests are made over HTTPS with a default HTTP/1.1 protocol, and the crawler supports gzip compression to reduce bandwidth usage. IP ranges are sourced from the ASN AS203478 (SeoStar Ltd.), and the bot routes through data centers in the US, EU, and APAC regions. The crawler adheres to RFC 7230 and includes a Referer header set to its root domain for traceability.

📋 robots.txt Compliance

SeoStar explicitly honors robots.txt Disallow directives and will pause crawling for any path marked as disallowed. The official documentation at seostar.io/robots-compliance confirms that the bot checks the file at each domain before initiating a crawl session. If a site returns a 503 Service Unavailable or 429 Too Many Requests, the bot backs off with an exponential-backoff algorithm, as verified in the project’s GitHub repository (commit a3f2e7d).

🔍 Detection Indicators

The primary User-Agent string is SeoStarBot/2.0 (+https://seostar.io/bot-info), with an alternative string SeoStar-Crawler/1.0 used for mobile-page validation. Behavioral fingerprints include a constant crawl window between 08:00–20:00 UTC and a request signature that includes the header X-SeoStar-ID: [hash]. Network administrators can verify the bot by querying the PTR record of its IPs, which resolves to crawler.seostar.io.

📊 Data Usage

Collected data—such as meta tags, heading structures, canonical URLs, and internal link counts—is used exclusively for generating custom SEO reports and competitor benchmarking tools within the SeoStar platform. The company’s privacy policy (published at seostar.io/privacy) states that no content is used for AI training or resold to third parties; it is retained for a maximum of 90 days and then anonymized.

⚙️ Rate Limiting Policy

While the bot is legitimate and follows ethical crawling practices, it is rate-limited to prevent excessive resource consumption on smaller sites. Threshold-based blocking (e.g., more than 100 requests per minute from a single IP) is recommended because SEO crawlers can inadvertently trigger DoS-like load on under-provisioned servers.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.