🤖 Overview

SalesIntelligent is a legitimate web crawler operated by SalesIntel, a sales intelligence and lead generation company based in the United States. According to the company’s official documentation at salesintel.com/robot, the bot’s primary purpose is to collect publicly available B2B contact and company information from websites, which feeds into SalesIntel’s proprietary database used by sales teams for prospecting and account-based marketing. The crawler was first publicly documented in 2018 and has since been updated to comply with evolving web standards.

🌐 Technical Behavior

The SalesIntelligent bot performs scheduled, moderate-frequency crawls, typically issuing requests at intervals of 10–30 seconds between pages to avoid server overload. It uses standard HTTP/1.1 and HTTPS protocols and respects Cache-Control headers. The crawler originates from IP ranges primarily belonging to Amazon Web Services (AWS) in the us-east-1 region, with occasional fallback to DigitalOcean and Hetzner, as confirmed by reverse DNS lookups published in community forums. The bot follows a breadth-first crawl strategy, starting from seed URLs submitted by customers or discovered from public directories.

📋 robots.txt Compliance

SalesIntel’s documentation explicitly states that SalesIntelligent fully honors robots.txt Disallow directives. Tests performed by webmasters and reported on Reddit and other forums confirm that the bot stops crawling any path listed in robots.txt within one crawl cycle. No documented cases of intentional disregard have been found; however, occasional delays in propagation (up to 48 hours) have been noted when robots.txt is updated.

🔍 Detection Indicators

The primary User-Agent string is Mozilla/5.0 (compatible; SalesIntelligent/1.0; +https://www.salesintel.com/robot). A secondary legacy string SalesIntelligent/1.0 is still observed on some older crawler instances. The bot also sends a custom header X-SalesIntel-Crawl: true and includes a From header containing a contact email (e.g., [email protected]) for abuse reports, as per their official policy.

📊 Data Usage

Collected data—including email addresses, phone numbers, job titles, and company descriptions—is parsed, deduplicated, and loaded into SalesIntel’s subscription-based sales intelligence platform. No content (e.g., full articles or images) is stored; only structured contact and metadata fields are retained. The data is used exclusively for B2B lead generation and is not employed for AI training or general search indexing.

⚙️ Rate Limiting Policy

Rate limiting of the SalesIntelligent bot is recommended because its moderate crawl frequency can still accumulate significant traffic over time, especially on sites with thousands of pages. A threshold-based block (e.g., >500 requests per hour from the same IP range) prevents resource exhaustion while allowing the legitimate lead-generation purpose to continue for most sites.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.