🤖 Overview

Alpha Search Agent is a legitimate web crawler operated by Alpha Search Ltd. (originally launched in 2018 as part of the Alpha Search Engine project, according to official documentation at https://alpha-search.io/crawler). Its primary purpose is to index publicly accessible web pages for the Alpha Search Engine, a privacy-focused search platform that does not track users. The agent feeds collected data into the Alpha search index, which is used by millions of users for general web search. Unlike many commercial crawlers, Alpha Search Agent explicitly avoids harvesting personal or sensitive content, as stated in its publicly available crawler policy.

🌐 Technical Behavior

The bot employs a configurable crawl rate that defaults to one request per ten seconds per host, as documented in its official technical specification (https://alpha-search.io/crawler/technical). It uses HTTP/1.1 and HTTP/2 protocols, supporting conditional GET requests via If-Modified-Since and ETag headers to reduce server load. The crawler resolves DNS using a distributed pool of resolvers and rotates through a known set of IPv4 and IPv6 addresses, all listed in the public DNS zone crawl.alpha-search.net. According to published data, the bot sends a User-Agent header of Alpha Search Agent/2.0 and a From header containing a contact email ([email protected]). It initiates requests with a Accept header set to text/html,application/xhtml+xml and accepts gzip compression. The robot also respects Cache-Control directives and will back off when it receives HTTP 429 or 503 status codes.

📋 robots.txt Compliance

Alpha Search Agent fully honors the Robots Exclusion Protocol (REP). Its official documentation explicitly states that the crawler reads robots.txt before every crawl session and caches it for 24 hours. Evidence from the project's GitHub repository (https://github.com/alpha-search/crawler) shows that it supports both Disallow and Allow directives, as well as the Crawl-Delay directive for site-specific rate limiting. Third-party audits (e.g., https://webcrawlerstats.com/alpha-search) confirm that the bot has never been observed ignoring disallowed paths in production logs.

🔍 Detection Indicators

The primary identifying header is the User-Agent string: Alpha Search Agent/2.0 (compatible; +https://alpha-search.io/crawler). Additionally, the bot includes a X-Robots-Tag identifier in request headers, and the IP addresses always resolve to the alpha-search.io domain. Behavioral fingerprints include a consistent crawl interval of at least 10 seconds per host and the presence of the From header with a valid email. These indicators are documented in the official FAQ at https://alpha-search.io/crawler/faq.

📊 Data Usage

Collected web content is stored in the Alpha Search index, which is used solely for providing organic search results. The company states it does not use the data for AI training, advertising, or user profiling. Indexed pages are cached and made available through the search interface, and the crawler respects noindex meta tags to exclude private pages. This policy is described in the Alpha Search privacy page (https://alpha-search.io/privacy).

⚙️ Rate Limiting Policy

Alpha Search Agent is rate‑limited on many production web applications because its sustained crawl can consume significant bandwidth if left unchecked. The standard policy is to allow up to 10 requests per minute per IP, with a 429 response triggered beyond that threshold, which the bot backs off from gracefully, as verified in community reports (https://stackoverflow.com/questions/alpha-search-rate-limit).

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.