🤖 Overview

Traazi is a web crawler operated by Traazi Inc., a data services company based in San Francisco, first publicly documented in early 2024. The bot is designed to systematically collect publicly accessible textual and structured web content for the purpose of training large language models and enhancing natural language processing pipelines within Traazi’s proprietary AI platform, known as TraaziCore. According to the official Traazi documentation published at docs.traazi.com/bot, the crawler adheres to a strict ethical framework and is explicitly not intended for any malicious activity.

🌐 Technical Behavior

Traazi employs a distributed crawl architecture using AWS and Google Cloud virtual machines, with IP ranges primarily belonging to AS16509 (Amazon), AS15169 (Google), and AS36351 (Microsoft Azure). The bot sends HTTP GET requests with a configurable delay of 5 to 15 seconds between consecutive requests to the same domain, as stated in the official robots.txt policy and verified by third-party monitoring tools. It supports both HTTP/1.1 and HTTP/2 protocols, and it requests HTML, JavaScript, CSS, and JSON resources while ignoring binary files unless explicitly allowed. The crawler also respects ETags and Last-Modified headers to avoid re‑downloading unchanged content, and it uses a custom Accept-Encoding header supporting gzip and brotli compression.

📋 robots.txt Compliance

Traazi fully honors Disallow directives found in robots.txt, as confirmed by Traazi’s code repository on GitHub (github.com/traazi/crawler-policy) and by webmasters who have reported accurate compliance. The crawler also respects Crawl-Delay directives and will back off to the specified delay interval. Any domain that explicitly blocks Traazi via Disallow is completely excluded from future crawls within the same session.

🔍 Detection Indicators

The primary User-Agent string is Mozilla/5.0 (compatible; TraaziBot/1.1; +https://www.traazi.com/bot). Additional identifying headers include X-Traazi-ID set to a unique session token and Via header showing the proxy origin. Behavioral fingerprints include sequential request patterns with a fixed delay and a strong preference for HTTP/2 over /1.1. The bot also sends a From header with a contact email address as specified in the documentation.

📊 Data Usage

Collected data is used exclusively for AI training within the TraaziCore language model, as well as for internal analytics to improve crawl efficiency. Traazi’s privacy policy states that no personal information (e.g., emails, login credentials) is stored or used; only anonymous, publicly available text and metadata are processed. The company publishes a data transparency report quarterly.

⚙️ Rate Limiting Policy

Traazi is rate-limited because its distributed architecture can generate a high volume of requests across many IPs simultaneously, potentially impacting server performance. The recommended policy is to allow the bot up to 20 requests per minute per IP and then temporarily block if thresholds are exceeded, ensuring fair resource usage for all visitors without treating the agent as malicious.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.