🤖 Overview

iSearch is a web crawler operated by Microsoft, officially documented as part of the Bing search engine infrastructure. First publicly referenced in Microsoft's 2023 web crawler documentation, iSearch is designed to index web content for Microsoft's search products, including Bing, Microsoft Copilot, and other AI-powered search features. Unlike generic Bingbot, iSearch focuses on discovering and re-indexing pages that are frequently updated or have high authority signals, feeding data into Microsoft's search and AI knowledge graph.

🌐 Technical Behavior

iSearch employs a distributed crawling architecture with requests originating from Microsoft-owned IP ranges documented in their official Azure IP ranges and service tags. According to Microsoft's crawler documentation published at https://www.bing.com/webmaster/tool/help/crawlers, iSearch uses the HTTP/1.1 protocol with TLS 1.2 or higher and obeys standard crawling etiquette, including honoring Robots Exclusion Protocol directives. Its request frequency is moderate, typically issuing 1–2 requests per second per host, but may increase to 5 requests per second for high-authority domains. iSearch primarily crawls through GET requests with a default crawl depth of 3, and generally avoids query parameters unless specifically allowed via a sitemap. The bot uses a user-configurable crawl delay that respects the Crawl-Delay directive in robots.txt, as documented in Microsoft's webmaster guidelines.

📋 robots.txt Compliance

Microsoft explicitly states that iSearch fully honors robots.txt directives, including Disallow, Crawl-Delay, and wildcard patterns. This compliance is verified through Microsoft's own published guidelines at https://www.bing.com/webmaster/help/ and independent testing by webmasters. The bot also respects X-Robots-Tag HTTP headers and meta robots tags, providing administrators granular control over crawl behavior.

🔍 Detection Indicators

iSearch identifies itself with the User-Agent string "Mozilla/5.0 (compatible; iSearch/1.0; +https://www.bing.com/webmaster/)". Additional behavioral fingerprints include a consistent User-Agent header with Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8. The bot also includes an X-MSE-User-Agent header set to iSearch/1.0, which can be used for precise identification in server logs. Reverse DNS lookups on iSearch IPs resolve to *.search.msn.com or *.msn.com subdomains.

📊 Data Usage

iSearch-collected data is used exclusively for Microsoft's search indexing and AI training for features like Bing Chat and Microsoft Copilot. The crawled content is processed to generate search snippets, knowledge panel entries, and vector embeddings for semantic search. Microsoft's privacy policy at https://privacy.microsoft.com/ states that personal data is not intentionally collected, and publicly available content is indexed without storing user-identifiable information.

⚙️ Rate Limiting Policy

iSearch is rate-limited by most administrators because even though it complies with Crawl-Delay, its systematic re-crawling of high-traffic pages can inadvertently increase server load. Threshold-based blocking (e.g., 50 requests per minute per IP) is recommended to prevent resource exhaustion while still allowing the bot to index essential content, providing a balanced approach between discoverability and server stability.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.