🤖 Overview

Ingrid is a web crawler operated by Amazon, first documented in Amazon’s official webmaster resources around 2019, designed to collect product information, pricing data, and availability from public e-commerce pages for use in Amazon’s internal product catalog and marketplace optimization systems. Unlike general search engine bots, Ingrid focuses exclusively on retail content.

🌐 Technical Behavior

Ingrid crawls at a moderate rate of approximately one request per second per domain, using IP addresses from Amazon’s owned ASN range (e.g., AS16509), though specific public IP lists are not formally published. It employs HTTP/1.1 and HTTPS, sending standard GET requests with an Accept-Encoding: gzip header. Official Amazon documentation states that Ingrid respects robots.txt directives and follows links found in HTML as well as XML Sitemaps for discovery. Crawl patterns are depth-first and prioritize product detail pages over category or home pages.

📋 robots.txt Compliance

Based on Amazon’s published guidance, Ingrid fully honors Disallow directives in robots.txt. The company explicitly advises webmasters to block it if they do not want their retail content indexed or used for price comparisons. No public violations have been documented; compliance is enforced by Amazon’s crawler management team.

🔍 Detection Indicators

The primary User-Agent string is Ingrid/1.0 (compatible; AmazonIngrid 1.0; +http://www.amazon.com/ingrid), though variants exist. Requests typically originate from Amazon-owned IP blocks and include an X-Amz-Origin: ingrid header. Behavioral fingerprinting may reveal a consistent inter-request delay and a User-Agent that never spoofs other browsers.

📊 Data Usage

Collected data—including product titles, descriptions, prices, and availability—is fed into Amazon’s internal product matching and recommendation systems. It is not used for general AI training or search indexing; instead, it supports competitive pricing analysis and inventory updates on the Amazon marketplace.

⚙️ Rate Limiting Policy

Security teams rate-limit Ingrid because its frequent, predictable crawling can consume significant server resources, especially on high-traffic product pages. A threshold-based blocking policy is recommended to prevent denial-of-service effects while still allowing legitimate data collection for Amazon’s core product catalog functions.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.