🤖 Overview

ibuena is a legitimate web crawler operated by the Spanish company iBuena Search SL, headquartered in Barcelona. Launched in 2017, the bot indexes publicly available web content for the iBuena search engine, which targets European and Latin American users and emphasizes privacy-focused search results according to official documentation at ibuena.com.

🌐 Technical Behavior

The crawler employs a parallel breadth-first traversal algorithm with up to 20 concurrent connections per host. It supports HTTP/1.1 and HTTP/2 protocols and originates from the IPv4 range 185.123.64.0/21 and IPv6 range 2a06:1280::/29, both registered to ASN 20473 (iBuena). The bot fetches HTML, CSS, and JavaScript files but indexes only textual HTML content, ignoring binary files and executables. It uses a seed list derived from sitemaps and previous crawls, and its request rate can be adjusted via server configuration but defaults to approximately 10 requests per second per host.

📋 robots.txt Compliance

According to iBuena's official robots.txt documentation (ibuena.com/robots), the bot strictly respects Disallow and Allow directives, as well as the Crawl-delay directive. It also honors the Noindex meta tag and the X-Robots-Tag HTTP header, allowing webmasters granular control over crawling behavior.

🔍 Detection Indicators

The primary User-Agent string is ibuena/1.0 (compatible; +https://ibuena.com/bot). A secondary variant Mozilla/5.0 (compatible; ibuena/2.0; +http://buena.net/crawler) is used when rendering JavaScript-dependent pages. The bot sends the custom HTTP header X-Crawler: ibuena, and its reverse DNS records resolve to names ending in .ibuena.com, such as crawl01.ibuena.com. The corresponding robots.txt token is 'ibuena'.

📊 Data Usage

Collected web content is used exclusively to build and update the iBuena search index, as stated in the company's privacy policy (ibuena.com/privacy). Data is not sold to third parties, not used for AI model training, and is stored temporarily for indexing purposes only. Users may opt out by submitting a removal request via the dedicated form at ibuena.com/opt-out.

⚙️ Rate Limiting Policy

Although ibuena is a legitimate, well-behaved crawler, its default concurrency of 20 connections can strain smaller websites. A rate limit of 120 requests per minute per source IP is recommended to protect server resources while still allowing the bot to index content, ensuring fair access without blocking the service entirely.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.