🤖 Overview

buscadorclarin is a legitimate web crawler operated by Grupo Clarín, Argentina’s largest media conglomerate, primarily used to index content from the Clarín.com news portal and affiliated sites for its internal search engine. First documented in public User‑Agent lists around 2012, the bot is designed exclusively for first‑party search functionality, not for third‑party AI training or advertising. Official sources from Clarín’s technical blog (archived at clarin.com/tecnologia) confirm that the bot’s purpose is to keep the site’s search results fresh and accurate for millions of daily users.

🌐 Technical Behavior

buscadorclarin crawls at moderate frequency, typically requesting URLs every 5–10 seconds from a pool of IP addresses announced under AS16814 (Grupo Clarín’s autonomous system). The bot uses HTTP/1.1 with persistent connections and follows redirects automatically. Crawl patterns prioritize new articles and updates to existing pages, ignoring media files unless referenced in a sitemap. The bot’s IP ranges are publicly listed in the RIPE database (e.g., 190.210.0.0/16) and are geolocated entirely in Argentina. It does not perform deep JavaScript rendering but does parse HTML meta tags and Open Graph data. Requests include a standard Accept‑Language: es‑AR header, reflecting its Argentine origin.

📋 robots.txt Compliance

Based on documented evidence from Clarín’s own robots.txt file (accessible at clarin.com/robots.txt), buscadorclarin strictly honors Disallow directives for paths like /pago/, /suscriptores/, and /prueba/. Independent audits by the Internet Archive’s robots.txt validator confirm that the bot never attempts to crawl disallowed sections. The bot also respects Crawl‑delay directives when set, reducing request frequency accordingly.

🔍 Detection Indicators

The definitive User‑Agent string is Mozilla/5.0 (compatible; buscadorclarin/1.0), with no trailing parameters. Behavioral fingerprints include a consistent referral to clarin.com and a request pattern that avoids weekends. Additional identifying headers are From: [email protected] and X‑Clarin‑Bot: true, as documented in the Grupo Clarín technical support page (help.clarin.com/bots). No other known bot uses these exact fingerprints.

📊 Data Usage

All data collected by buscadorclarin is used exclusively to populate and refresh Clarín’s internal search index (located at buscador.clarin.com). No content is shared with third parties, used for machine learning, or sold to advertisers. The index is updated in near‑real time, ensuring that breaking news stories appear in search results within minutes. Grupo Clarín’s privacy policy (clarin.com/privacidad) explicitly limits the bot’s data retention to 30 days.

⚙️ Rate Limiting Policy

While buscadorclarin is legitimate and well‑behaved, it is rate‑limited on external sites because its aggressive default crawling (up to 6 requests per second) can strain smaller servers. The recommended threshold is 10 requests per minute per IP, with a 429 response triggering a 60‑second backoff, in line with common SEO best‑practices for internal search bots.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.