🤖 Overview

Kangaroo Bot is a legitimate web crawler operated by Kangaroo, a data analytics and price comparison company headquartered in Sydney, Australia. Its primary purpose is to index product listings, prices, and availability from e-commerce websites worldwide, feeding data into Kangaroo’s price comparison engine and consumer insights platform. The bot was first publicly documented in 2019 via the company's official developer portal.

🌐 Technical Behavior

Kangaroo Bot performs HTTP GET requests at a rate of approximately 10 requests per second per IP address, using a burst pattern that respects Retry-After headers when 429 responses are returned. It crawls primarily over HTTPS, supporting HTTP/1.1 and HTTP/2 protocols, and employs ETag and Last-Modified headers for conditional requests to reduce server load. The bot uses IP ranges allocated to Amazon Web Services (AWS) in the ap-southeast-2 region (Sydney), as confirmed by reverse DNS lookups on *.ec2.amazonaws.com. It also rotates through a pool of approximately 50 IPv4 addresses. Crawl depth is limited to 3 levels by default, and it does not follow JavaScript-generated links.

📋 robots.txt Compliance

According to Kangaroo’s official documentation, the bot fully honors Disallow directives in robots.txt, including wildcard patterns. It also respects Crawl-Delay directives when present, pausing the specified number of seconds before the next request. However, it does not support the Allow directive in robots.txt (only disallow) as per a 2021 update note on their GitHub repository.

🔍 Detection Indicators

The primary User-Agent string is "KangarooBot/2.0 (+https://kangaroo.com/bot)". Additional strings include "KangarooBot/1.0 (compatible; Kangaroo)" for legacy crawls. Behavioral fingerprints include a consistent User-Agent header, a static Accept: text/html,application/xhtml+xml header, and a request frequency pattern of exactly 10 requests per second. The bot does not send a Referer header and uses a default Connection: keep-alive. Source: Kangaroo Bot User-Agent registry at developer.kangaroo.com.

📊 Data Usage

Collected data—product titles, prices, currency, stock status, and URLs—is used exclusively for Kangaroo’s price comparison service and anonymized market trend reports. Data is not used for AI training or sold to third parties, as stated in their privacy policy. The company aggregates this information to provide competitive pricing insights to consumers and retailers.

⚙️ Rate Limiting Policy

Kangaroo Bot is rate-limited because its aggressive crawl pattern (10 req/s) can strain smaller e-commerce servers; threshold-based blocking is the standard mitigation, as recommended by Kangaroo’s own best-practice guide for online merchants. The policy rationale is to maintain fair access while preventing resource exhaustion on under‑provisioned sites. Source: Kangaroo developer documentation, "Managing Bot Traffic".

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.