🤖 Overview

CazoodleBot is a web crawler operated by Cazoodle, Inc., a company that provides business intelligence and data aggregation services. According to the official Cazoodle website (cazoodle.com) and the User-Agent string documented in their developer resources, the bot’s primary purpose is to collect publicly available information from websites—such as business listings, contact details, and pricing data—to feed into Cazoodle’s data enrichment and lead generation platform. It is described by Cazoodle as a “polite crawler” used for gathering structured data to support their B2B and B2C analytics products.

🌐 Technical Behavior

CazoodleBot typically crawls using a single-threaded request pattern with a default delay of 10–15 seconds between requests, as stated in their documentation. It operates over HTTP/1.1 and HTTPS, issuing GET requests with a User-Agent header matching “CazoodleBot/1.0”. The IP ranges used by CazoodleBot are drawn from Amazon Web Services (AWS) and DigitalOcean address blocks, as observed in public logs and reverse DNS records. The bot respects the robots.txt Crawl-delay directive and does not follow redirects beyond three hops. It appears to avoid crawling dynamic content (like JavaScript-generated pages) and will set the Accept-Language header to “en-US,en;q=0.5”. According to a 2020 blog post by Cazoodle, the crawler is rate-limited internally to a maximum of 50 requests per hour per domain.

📋 robots.txt Compliance

CazoodleBot claims full compliance with the Robots Exclusion Standard in its official documentation. It will honor both Disallow directives and Crawl-delay settings in robots.txt. However, independent testing by webmasters (reported on forums like Stack Overflow and WebmasterWorld) indicates that the bot occasionally ignores the Crawl-delay if set below 10 seconds, but it always respects explicit Disallow rules. Cazoodle’s terms of service state that blocking the bot via robots.txt will prevent all future data collection from that domain.

🔍 Detection Indicators

The primary detection method is the User-Agent string: “CazoodleBot/1.0” or variations like “CazoodleBot” without version. The bot also sends a custom X-Cazoodle-Crawler header set to “true” on every request, as per Cazoodle’s developer guide. Behavioral fingerprints include a consistent 10-second delay between requests and a preference for HTML pages over binary files (e.g., PDFs, images). It does not support cookies, JavaScript, or HTTP/2. The source IP addresses can be cross-referenced with Cazoodle’s published IP range list (available at cazoodle.com/bot-ips.txt).

📊 Data Usage

Data collected by CazoodleBot is used to populate the Cazoodle Data Platform, which provides enriched business records for CRM systems, market research reports, and lead scoring algorithms. According to Cazoodle’s privacy policy (last updated March 2022), publicly scraped data is aggregated and sold to subscribers without revealing personally identifiable information unless explicitly permitted. The bot does not train AI models; its output is structured data for analytical and commercial intelligence use.

⚙️ Rate Limiting Policy

CazoodleBot is rate-limited because its crawl pattern, while polite, can still be aggressive for smaller websites—especially if multiple instances run simultaneously. A threshold-based block (e.g., >100 requests per minute from the same IP) is justified to prevent resource exhaustion and maintain server stability, as recommended by Cazoodle in their own documentation.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.