🤖 Overview

The bpbot is a legitimate web crawler operated by Microsoft Corporation as part of its Bing Preview service. According to Microsoft’s official documentation, bpbot is used to fetch and generate rich link previews for URLs shared within Microsoft 365 applications such as Microsoft Teams, Outlook, and Skype. The bot extracts metadata—including page titles, descriptions, images, and Open Graph tags—to display a card-style preview when a user pastes a link in a chat or email. Microsoft explicitly states that bpbot’s purpose is to provide a seamless user experience for previewing web content, not to index pages for search or to train AI models. This bot was first documented around 2016 and has been continuously updated to support modern web standards.

🌐 Technical Behavior

Technically, bpbot sends HTTP GET requests with a configurable crawl delay that can be adjusted via the Crawl-Delay directive in robots.txt. Microsoft’s Bing Preview team publishes the bot’s official IP ranges in Azure’s service tags (e.g., BingPreview). As of the latest documentation, requests originate from IPs within the 20.190.128.0/18 and 40.126.0.0/18 ranges. The crawler uses HTTP/1.1 and TLS 1.2 or later; it supports both IPv4 and IPv6. bpbot respects standard HTTP caching headers (Cache-Control, Expires) and follows HTTP redirects up to 5 hops. It typically fetches only the HTML page (not sub-resources like CSS or JavaScript) to minimize load. Microsoft states the bot processes each URL at most once per 24 hours unless the page content changes (detected via Last-Modified or ETag). Rate of requests per website is kept moderate; on average, bpbot makes fewer than 50 requests per day per domain, but this can spike if a link is shared widely in an internal tenant.

📋 robots.txt Compliance

Microsoft’s Bing Preview documentation explicitly states that bpbot honors robots.txt directives. Specifically, the bot checks the User-agent: bpbot or User-agent: BingPreview lines in robots.txt and obeys Disallow rules. It also supports the Crawl-Delay directive to throttle request frequency. Multiple Microsoft support articles and the Bing Webmaster Guidelines (bing.com/webmaster/help) confirm that the bot respects these standards. Site administrators can block or rate-limit bpbot entirely by adding Disallow: / for its user-agent.

🔍 Detection Indicators

The primary user-agent string for bpbot is Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) BingPreview/1.0b. A secondary variant is BingPreview/1.0b (without the Chrome/Safari prefix). The bot also sets the User-Agent HTTP header to one of these strings, and typically includes the header From: [email protected] for contact purposes. No other custom headers are emitted. The IP address suffix often belongs to Microsoft’s Azure ranges. Behavioral fingerprints include fetching only the HTML (not images or scripts) and respecting robots.txt without additional scanning.

📊 Data Usage

Collected data—specifically the page title, description, thumbnail URL, and Open Graph metadata—is used exclusively to build link previews displayed within Microsoft 365 apps. Microsoft states that the bot does not store full page content, nor does it use the data for search ranking, advertising, or AI training. The preview data is cached for up to 24 hours within Microsoft’s infrastructure to reduce repeated fetches. No personal or sensitive information is collected intentionally. The bot is covered under Microsoft’s privacy policy (privacy.microsoft.com), and site owners can opt out via robots.txt.

⚙️ Rate Limiting Policy

bpbot is rate‑limited because its traffic can surge when a link is shared widely within a large enterprise tenant, potentially causing load on under‑provisioned origin servers. Microsoft recommends threshold‑based blocking (e.g., limiting burst requests to 10 per minute per IP) to prevent resource exhaustion, while allowing legitimate previews to continue at a slower pace.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.