🤖 Overview

Info Seeker is a web crawler operated by InfoSeeker Technologies, a data analytics firm based in San Francisco. First publicly documented in 2018, the bot systematically collects publicly accessible web content to feed into InfoSeeker’s proprietary market intelligence and trend analysis platform. According to the official website infoseeker.com, the bot’s purpose is to gather structured and unstructured data for enterprise clients.

🌐 Technical Behavior

The bot uses a breadth-first crawl strategy with a configurable depth, typically up to 10 levels from the seed URL. It sends HTTP/1.1 requests with Keep-Alive headers and supports gzip compression and ETag-based caching. Average request frequency is 2–3 requests per second per domain, but it may burst up to 10 requests per second for short intervals. IP addresses are drawn from AWS (us-east-1, us-west-2) and Google Cloud (us-central1) ranges, documented in a published netblock list at infoseeker.com/ip-ranges. The bot respects Rel="nofollow" meta tags and Link headers for pagination discovery.

📋 robots.txt Compliance

According to the official robots.txt specification at infoseeker.com/bot-policy, Info Seeker fully honors Disallow directives. It checks for a dedicated User-agent: InfoSeeker line and falls back to User-agent: * if absent. Webmaster feedback on forums like WebmasterWorld confirms it respects Crawl-Delay directives with a granularity of seconds.

🔍 Detection Indicators

The primary User-Agent string is Mozilla/5.0 (compatible; InfoSeeker/1.0; +http://www.infoseeker.com/bot). A secondary string InfoSeeker/2.0 (compatible; +http://www.infoseeker.com/bot2) was introduced in 2020. The bot also sends a custom HTTP header X-InfoSeeker-ID with a unique alphanumeric identifier. IP addresses can be verified against the published netblock list.

📊 Data Usage

Collected data is used to train InfoSeeker’s AI models for market trend prediction, competitor pricing analysis, and brand sentiment monitoring. The data is aggregated, anonymized, and sold as part of a subscription-based analytics dashboard. It is not used for general search indexing or advertising targeting.

⚙️ Rate Limiting Policy

Although legitimate, Info Seeker’s burst traffic can degrade server performance. A rate limit of 5 requests per second per IP with a 10-minute sliding window is recommended, with threshold-based blocking if requests exceed 1,000 per hour from a single IP. This preserves site availability while allowing the bot to complete its crawl within reasonable time.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.