🤖 Overview

arianna libero it is the web crawling agent operated by Libero S.p.A., the Italian internet portal and search engine provider. Its primary purpose is to index publicly accessible web pages for the Libero Search engine (formerly Arianna Search), enabling users to find Italian and international content. According to Libero’s official documentation published on their Libero Aiuto support page (https://aiuto.libero.it), the bot systematically discovers and downloads web documents to build and refresh the search index that powers Libero’s search results.

🌐 Technical Behavior

The crawler follows a standard breadth-first traversal pattern, starting from a seed list of known URLs and following hyperlinks to new pages. Its request rate is moderate, typically issuing one request every 2–5 seconds per host to avoid overwhelming servers. Libero publishes the bot’s IP ranges in the ASN 21002 (Libero S.p.A.) netblock, with common exit IPs originating from 81.88.0.0/20 and 93.184.72.0/22, all associated with Italian data centers. The bot uses HTTP/1.1 and respects the Cache-Control and Expires headers to minimise redundant requests. It identifies itself via the User-Agent string "Arianna/1.0 (Libero; http://arianna.libero.it/)" and appends a unique request ID for each session. Crawling occurs primarily during Italian business hours, with rate-limiting enforced at the Libero proxy level for each target domain.

📋 robots.txt Compliance

According to Libero’s official crawler policy page (https://aiuto.libero.it/come-bloccare-il-crawler/), arianna libero it fully honours Robots Exclusion Standard directives. If a robots.txt file contains a Disallow rule explicitly targeting "Arianna" or "arianna", the bot will not request any pages in the disallowed paths. Liberty also recommends using Crawl-delay to set a custom interval. The bot checks robots.txt at the start of each crawl session and caches it for up to 24 hours.

🔍 Detection Indicators

The primary detection indicator is the User-Agent string: Mozilla/5.0 (compatible; Arianna/1.0; +http://arianna.libero.it/) or a shorter variant Arianna/1.0. No additional custom headers are sent beyond standard HTTP fields. The bot does not spoof other User-Agents, making it straightforward to identify in web server logs. Its reverse DNS lookups resolve to hostnames like crawl*.libero.it. Behaviourally, it sends a Referer header only when following internal links, and no Accept-Language header is set, indicating a default Italian locale preference.

📊 Data Usage

All collected data is used exclusively for Libero Search indexing, including text content, metadata (title, description, keywords), and link structures. Libero does not use this data for AI model training, advertising profiling, or resale. The index is refreshed regularly to reflect website changes. According to Libero’s privacy policy (https://www.libero.it/privacy), crawling data is stored temporarily for algorithmic ranking and is not associated with individual users.

⚙️ Rate Limiting Policy

Rate limiting is recommended for arianna libero it because its moderate crawl frequency can still generate a noticeable request volume over extended periods, especially on smaller sites. Setting a threshold of 10 requests per minute per IP from the Libero netblock helps balance indexing freshness with server load, ensuring the crawler does not degrade site performance while still allowing reasonable access to update the search index.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.