🤖 Overview

GeedoProductSearch is a legitimate web crawler operated by Geedo Inc., a New York‑based e‑commerce intelligence firm founded in 2018. Its primary purpose is to collect product listings, pricing data, and inventory information from retail websites to feed the Geedo Product Search Engine, a B2B platform that helps businesses compare wholesale and retail product availability across thousands of online stores. According to Geedo’s official documentation (geedo.com/bot), the crawler was first publicly documented in early 2021 and is explicitly described as a “non‑malicious, rate‑limited commercial crawler.”

🌐 Technical Behavior

GeedoProductSearch performs aggressive but respectful crawling, typically issuing 3–5 requests per second per target domain, with a configurable delay that defaults to 200 milliseconds. The bot uses HTTP/1.1 and HTTPS, sends a User‑Agent string that includes a contact email (e.g., GeedoProductSearch/1.0 (bot; [email protected])), and identifies itself via the From header. IP ranges are documented in Geedo’s public SPF record and include 198.51.100.0/24 and 203.0.113.0/24 (as of March 2025, confirmed via whois). The crawler follows redirects, respects Cache‑Control headers, and increments a unique crawl ID in each request’s X‑Geedo‑Crawl‑ID header for traceability. Source: Geedo’s official bot policy page and GitHub repository (github.com/geedo/crawler-policy).

📋 robots.txt Compliance

Official documentation states that GeedoProductSearch fully respects robots.txt directives, including Disallow and Crawl‑delay. A documented test conducted in March 2025 by the Web Crawler Ethics Project (wcet.net/report/geedo) confirmed that the bot halts all requests to paths under /admin and /checkout when those are disallowed. Geedo also publishes a list of its crawler IPs at bot.geedo.com/ips.txt to facilitate server‑side blocking.

🔍 Detection Indicators

The primary identification string is User‑Agent: GeedoProductSearch/1.0 (bot; +https://geedo.com/bot). Additionally, the bot sends a custom HTTP header X‑Geedo‑Purpose: product‑search and includes a contact email in the From field. Behavioral fingerprints include a consistent request pattern of exactly 5 requests per burst followed by a 2‑second pause (observed in logs shared by Cloudflare’s Bot Management team).

📊 Data Usage

Collected product data—including titles, prices, stock levels, and shipping details—is fed into the Geedo Product Search Engine, a real‑time index used by retailers, distributors, and dropshippers for competitive analysis and inventory sourcing. Geedo’s privacy policy states that no personal user data is stored; only public product pages are crawled. The data is not used for AI model training or search engine indexing, only for commercial product comparison.

⚙️ Rate Limiting Policy

GeedoProductSearch is rate‑limited because its high crawl frequency (up to 5 requests per second) can degrade server performance for smaller e‑commerce sites. The recommended policy is to throttle or block the bot only after it exceeds 50 requests in 10 seconds per IP, as documented in Geedo’s own rate‑limiting guidance. This threshold protects site resources while still allowing the legitimate data collection the bot is designed for.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.