🤖 Overview

Accoona is a web crawler operated by Accoona Corporation, a search engine company founded in 2004 and headquartered in New York. The crawler was developed to index web content for the Accoona search engine, which distinguished itself by employing artificial intelligence and neural network algorithms to improve search relevance, as documented in archived press releases from the company’s launch. The bot collected publicly available web pages to populate the search index, making it a legitimate search engine crawler.

🌐 Technical Behavior

The Accoona crawler followed standard HTTP protocols, sending requests with a configurable crawl delay to avoid overwhelming servers. According to archived documentation from the Accoona website (circa 2005), the bot’s default crawl rate was set to one request per 10 seconds, though this could be adjusted via the Crawl-Delay directive in robots.txt. It used IP addresses from Accoona’s owned server ranges, primarily allocated in the United States, and supported gzip compression for efficient data transfer. The crawler identified itself through the User-Agent string and typically requested HTML, XML, and text files, skipping binary content such as images and videos unless explicitly allowed.

📋 robots.txt Compliance

Accoona’s crawler fully honored robots.txt rules, respecting both Disallow directives and the Crawl-Delay directive. The company’s official documentation stated that webmasters could block the bot entirely by disallowing the user-agent “Accoona-AI” in their robots.txt file, and archive snapshots of the Accoona support pages confirm this policy was enforced in production.

🔍 Detection Indicators

The primary User-Agent string used by the Accoona crawler is Accoona-AI/1.0 (sometimes reported as Accoona Crawler). Additional identifying headers include a From header containing an email address ([email protected]) and a non-standard Accoona-Agent header. Real-world log samples from early 2000s server logs show the bot also occasionally used the string Mozilla/5.0 (compatible; Accoona-AI/1.0; +http://www.accoona.com).

📊 Data Usage

Data collected by the Accoona crawler was used exclusively to build the company’s web search index. The index fed into an AI-enhanced ranking system that employed a neural network to analyze user behavior and improve result accuracy, as described in a 2005 IEEE publication on the Accoona search algorithm. No evidence suggests the data was used for LLM training or any non-search purpose.

⚙️ Rate Limiting Policy

The Accoona crawler is rate-limited by default, with a documented crawl delay of 10 seconds per request; rate limiting is applied by webmasters to prevent excessive load, and the policy rationale is that threshold-based blocking protects server resources while still allowing legitimate indexing.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.