🤖 Overview

Poirot is a web crawler operated by Poirot sp. z o.o., the Polish company behind the poirot.pl search engine. First launched in 2004, the bot systematically indexes publicly accessible web pages to populate and refresh the Poirot search index, which serves Polish-language and global internet users. The product is a dedicated search engine competing with Google, Bing, and local alternatives, focusing on privacy and localized results.

🌐 Technical Behavior

The PoirotBot crawler uses standard HTTP GET requests with a default crawl rate of approximately one request per second per host, though it may adjust based on server response times. It respects the Robots Exclusion Protocol by checking the robots.txt file before each crawl session (documented on the official poirot.pl/bot page). The crawler originates from IPv4 addresses primarily in Poland, announced under the ASN attributed to Poirot (AS197109), with ranges such as 91.192.0.0/20 and 93.105.0.0/16, though these may change. It uses HTTP/1.1 with Accept-Encoding: gzip and Connection: keep-alive headers.

📋 robots.txt Compliance

According to the official poirot.pl/bot documentation, PoirotBot fully honors Disallow directives in robots.txt. It also respects Crawl-Delay directives when present. No known violations or CVE entries associated with robots.txt non-compliance have been reported.

🔍 Detection Indicators

The primary User-Agent string is PoirotBot/1.0 or Mozilla/5.0 (compatible; PoirotBot/1.0; +http://poirot.pl/bot). Behavioral fingerprints include a consistent request pattern with Accept: text/html,application/xhtml+xml and a missing Referer header. The bot does not set cookies and uses a static From header rarely.

📊 Data Usage

Collected data is used exclusively for web search indexing in the Poirot search engine. The index supports keyword searches, page ranking, and snippet generation. Poirot explicitly states that crawling data is not used for AI model training, targeted advertising, or sold to third parties, as noted on their privacy policy page.

⚙️ Rate Limiting Policy

While PoirotBot is legitimate and generally well-behaved, it can become aggressive during large-scale re-crawls. Rate limiting is enforced by webmasters to prevent resource exhaustion—threshold-based blocking (e.g., >10 requests per second per IP) is recommended, with the policy balancing fair access against server stability.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.