🤖 Overview

Sondeur is a legitimate web crawler operated by the French company Sondeur Analytics, launched in 2019 to provide continuous SEO monitoring and website audit services. Its primary purpose is to collect publicly accessible HTML, metadata, and resource files (CSS, JavaScript) for indexing into the Sondeur analytics platform, which delivers real‑time performance and ranking insights.

🌐 Technical Behavior

Sondeur initiates crawl sessions using both HTTP/1.1 and HTTP/2, sending a User-Agent header that includes its version and contact URL. According to the official documentation at sondeur.com/crawler, its default crawl rate is 10 requests per second, configurable via the Crawl-Delay directive in robots.txt. The bot operates from a fixed IP range announced under ASN 207012: 185.12.64.0/22. Requests are made with a typical spread of 1–5 seconds between pages, but the bot may burst up to 15 requests in a few seconds when encountering redirect chains.

📋 robots.txt Compliance

Sondeur explicitly honors the Disallow and Crawl-Delay directives as documented on its sondeur.com/robots.txt page. It requires a dedicated user‑agent token Sondeur; without a rule, it follows standard generic directives. The bot does not crawl URLs found in sitemap.xml that are excluded by Disallow rules, confirming its compliance.

🔍 Detection Indicators

The primary User‑Agent string is Mozilla/5.0 (compatible; Sondeur/1.0; +https://sondeur.com/bot), and an alternative SondeurBot/1.0 is used for mobile‑first crawls. Behavioral fingerprints include a mandatory From header set to [email protected] and a custom X‑Sondeur‑Crawl header with a unique session ID. IPs resolve to the sondeur‑crawl hostname prefix.

📊 Data Usage

Collected data (page titles, meta descriptions, headings, image alt text, HTTP headers) is aggregated into the Sondeur platform for SEO dashboards, backlink reports, and competitor benchmarking. No personal or sensitive content is stored; the data is used exclusively for statistical and analytical purposes, not for AI training or model development.

⚙️ Rate Limiting Policy

Rate limiting is recommended because Sondeur’s default crawl speed of 10 req/s can overwhelm under‑provisioned servers or cause significant load during concurrency. Setting a throttle of 2 requests per second via Crawl-Delay: 5 or a web application firewall rule ensures consistent access without degrading site performance.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.