🤖 Overview

Najdi.si is the primary web crawler operated by the Slovenian search engine Najdi.si, a subsidiary of the media company TSmedia. It is designed to index Slovenian and regional web content to power the najdi.si search portal, which serves as the leading search engine in Slovenia. According to the official website (najdi.si), the crawler collects publicly accessible pages to provide relevant search results for users in the region.

🌐 Technical Behavior

The Najdi.si crawler, identified by the user-agent string najdi.si (often without version numbers), follows standard HTTP protocols and respects robots.txt directives. Based on publicly available logs and community reports, it typically requests pages at a moderate rate of a few requests per second, avoiding server overload. The crawler originates from IP ranges registered to the Slovenian ISP Telekom Slovenije (AS5603) and other local providers, but specific ranges are not officially published. It indexes both HTML and linked resources such as images and CSS, primarily for search ranking rather than AI training.

📋 robots.txt Compliance

According to the official robots.txt documentation provided by Najdi.si (available at help.najdi.si), the crawler fully honors Disallow directives as defined in the Robots Exclusion Protocol. It also respects crawl-delay directives if specified, making it well-behaved for site administrators.

🔍 Detection Indicators

The primary identifying user-agent string is najdi.si (case-insensitive). No additional custom headers or IP-specific patterns are documented. Web servers can detect it by matching the user-agent header in HTTP requests; it does not disguise itself as a different browser.

📊 Data Usage

Collected data is used exclusively to build and maintain the Najdi.si search index, enabling users to find Slovenian and regional websites. There is no evidence that the data is repurposed for AI training, advertising profiling, or third-party sharing. The search engine is primarily a local service with a privacy-focused approach.

⚙️ Rate Limiting Policy

Rate limiting for this bot is recommended to prevent excessive load on servers, especially for small sites, though its typical crawl rate is modest. A threshold-based blocking policy (e.g., >10 requests per second) is appropriate to distinguish aggressive misconfigurations from the normal behavior of this legitimate regional crawler.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.