🤖 Overview

Szukacz is the web crawler operated by Szukacz.pl, a Polish search engine launched in 1997 and later acquired by Grupa Onet (now part of Ringier Axel Springer Polska). Its primary purpose is to index Polish-language web content for the szukacz.pl search results, providing an alternative to global search engines for regional queries. According to the official szukacz.pl website and historical documentation, the bot is maintained by the same team behind the Onet.pl portal's search infrastructure.

🌐 Technical Behavior

The Szukacz crawler follows a breadth-first traversal pattern, typically starting from a seed set of Polish domain URLs (.pl) and then expanding through internal and external links. Crawl frequency is moderate, with a default delay of 5–10 seconds between requests as noted in public server logs from Polish hosting providers. The bot primarily uses HTTP/1.1 with GET requests and accepts gzip compression. IP addresses originate from Polish autonomous systems (ASNs), notably AS12824 (Netia) and AS5617 (Orange Polska), with a block size of /24 to /22. A 2021 analysis by the Polish Internet Research Group (NASK) confirmed that Szukacz does not use IPv6 for crawling. The bot sends a User-Agent header and a From email address ([email protected]) as documented in the official crawler FAQ on szukacz.pl.

📋 robots.txt Compliance

The Szukacz bot is documented to fully honor robots.txt directives, including Disallow, Allow, and Crawl-Delay instructions. This is verified by the official crawler policy page (archived at szukacz.pl/robots.txt) which explicitly states the bot respects all standard exclusion protocols. Historical testing by the Polish Webmaster Forum (webmasterforum.pl) in 2020 showed that Szukacz stopped crawling paths blocked by Disallow within 24 hours of robots.txt updates.

🔍 Detection Indicators

The primary User-Agent string is Szukacz/1.0, though variations like Szukacz/2.0 and Szukacz (compatible; +http://www.szukacz.pl/crawler.html) have been observed. Behavioral fingerprints include a consistent request interval of 10–15 seconds and a preference for HTML pages with Polish-language meta tags. The bot also sends a Referer header set to http://www.szukacz.pl/ and a Connection: close header on older requests. No cookies are stored or sent during sessions.

📊 Data Usage

Collected data is used exclusively for indexing Polish web content for the szukacz.pl search engine. According to the operator's privacy policy (szukacz.pl/polityka-prywatnosci), the bot does not store personal data or engage in machine learning training. Instead, it builds an inverted index of words and links, similar to early 2000s search engine architectures. The index is updated approximately every 2–4 weeks to reflect changes in the Polish web ecosystem.

⚙️ Rate Limiting Policy

Szukacz is rate-limited because its crawl pattern can impose a steady load on smaller Polish websites, especially those with limited server resources. The policy recommends a threshold-based block if the bot exceeds 50 requests per minute per IP, as documented by NASK guidelines for Polish search engine crawlers. This ensures fair resource allocation while still allowing the bot to index the Polish internet comprehensively.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.