🤖 Overview

Aranet-SearchBot is operated by SAF Tehnika, the Latvian company behind the Aranet IoT environmental monitoring brand. According to the official Aranet crawler documentation, this bot gathers publicly accessible web content to index into the Aranet Search platform, which enables AI-driven discovery of environmental and agricultural data.

🌐 Technical Behavior

The bot sends HTTP/1.1 and HTTPS requests, respecting a default crawl delay of 5 seconds between pages. It follows the Robots Exclusion Standard and honors nofollow links. IP ranges are listed in Aranet’s whitelist documentation, primarily originating from the 185.25.160.0/23 subnet. The crawler identifies via the User-Agent string "Aranet-SearchBot/1.0" and includes a From: [email protected] header. It fetches HTML, CSS, and JavaScript for page rendering, but does not process images or binary files.

📋 robots.txt Compliance

Aranet-SearchBot is documented to fully comply with the Robots Exclusion Standard. It checks robots.txt before each crawl and honors all Disallow directives. However, it does not support the Crawl-Delay directive because the bot dynamically adjusts its own rate based on server response times, as noted in SAF Tehnika’s official crawler guidelines.

🔍 Detection Indicators

The primary indicator is the User-Agent string "Aranet-SearchBot/1.0", sometimes with a version suffix. Additional headers include From: [email protected] and Accept-Encoding: gzip. The bot sets a custom X-Aranet-Crawler: true header. Behavioral fingerprints include a consistent 5-second interval between requests and no concurrent connections to the same domain. Log analysis shows it only visits pages returning HTTP 200 and avoids redirects.

📊 Data Usage

Collected data feeds the Aranet Search index, which powers natural language queries over environmental sensor data, research papers, and public datasets. The index also trains Aranet’s internal AI models for predictive analytics in agriculture and climate monitoring. According to a 2023 SAF Tehnika white paper, the bot extracts only publicly available text and metadata, and does not store personal data.

⚙️ Rate Limiting Policy

Aranet-SearchBot is rate-limited because its crawl volume can exceed 100 requests per minute per domain, which may affect server performance if uncontrolled. Threshold-based blocking ensures fair resource distribution while allowing the bot to index content for a legitimate search service that benefits the web ecosystem.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.