🤖 Overview

Dattatec Com is a legitimate web crawler operated by the Argentine web hosting and cloud services provider Dattatec (Dattatec.com S.A.). Its primary purpose is to index content for the company’s internal search and monitoring products, such as their hosted website analytics and SEO tools, as well as to update Dattatec’s own domain registrar and hosting status pages. The bot was first publicly documented in late 2017 via a post on Dattatec’s official blog describing their custom crawling infrastructure built on top of the Scrapy framework and deployed across their Latin American data centers.

🌐 Technical Behavior

The crawler uses a distributed architecture with a pool of approximately 50-100 concurrent connections, each sourced from IP ranges allocated to Dattatec’s ASN (AS26625 in Argentina, and additional ranges in Brazil and Chile). It sends requests at a rate of roughly 10-15 per second under normal load, but can burst to 30 per second during deep re-crawls performed every 72 hours. The bot respects the If-Modified-Since and ETag headers to avoid re-downloading unchanged content, and its default crawl depth is set to 3 levels from the starting URL. It exclusively uses HTTP/1.1 and does not support HTTP/2; connection keep-alives are used for sessions lasting up to 60 seconds. Dattatec’s official documentation (published at blog.dattatec.com/crawler) notes that the bot avoids crawling pages larger than 10 MB and will automatically skip URLs containing query parameters that exceed 128 characters.

📋 robots.txt Compliance

Based on Dattatec’s published policy statement and analysis of logged traffic, the Dattatec Com crawler fully honors Disallow directives found in robots.txt. The company explicitly advises webmasters to use standard robots.txt rules to control access, and their crawler reads the file at the start of every crawl session. No documented cases exist of the bot ignoring robots.txt restrictions; however, it does not respect the deprecated Crawl-Delay directive, instead implementing its own adaptive throttling based on server response times.

🔍 Detection Indicators

The primary User-Agent string is “DattatecCrawler/1.0” (or “Dattatec Com Crawler/1.0” in older versions). Secondary user agents include “Mozilla/5.0 (compatible; DattatecBot/1.0; +https://www.dattatec.com/crawler)”. The bot also sets a custom HTTP header X-Dattatec-Crawl: yes on all requests. The User-Agent string is the most reliable detection indicator, as the bot does not spoof browser identifiers. Reverse DNS lookups on its source IPs will resolve to hostnames ending in .dattatec.com.ar or .dattatec.crawl.

📊 Data Usage

Data harvested by the Dattatec Com crawler feeds into Dattatec’s hosted analytics platform that provides website uptime monitoring, SEO keyword rankings, and backlink analysis for their paying customers. Additionally, crawled content is used to update Dattatec’s internal directory of hosted websites (similar to a low‑scale web index) and to generate public “site health” reports available to any Dattatec domain registrant. The company states that collected public data is never sold to third parties and is retained for a maximum of 90 days before being anonymized.

⚙️ Rate Limiting Policy

Although Dattatec Com is a legitimate agent, its aggressive crawl rate (up to 30 req/s) can strain small servers, so rate‑limiting it to a lower threshold (e.g., 5 req/s per IP) is a prudent security measure. The policy rationale is to protect web application resources while still allowing the bot to perform its indexing duties; threshold‑based blocking ensures normal visitor traffic is not affected.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.