🤖 Overview

ZumBot is a web crawler operated by Zum (formerly Zum.com), a search engine and AI‑powered knowledge platform based in San Francisco, USA. Its primary purpose is to index publicly accessible web content to populate Zum’s search index and to train its proprietary language models, as documented on the official Zum developer portal (https://developers.zum.com/bots). Unlike general‑purpose search crawlers, ZumBot focuses on high‑quality, authorative sources but does not restrict itself to any specific domain categories.

🌐 Technical Behavior

ZumBot issues HTTP/1.1 requests with a typical frequency of 1 request every 2–3 seconds per domain, respecting a crawl delay of 5 seconds when specified in robots.txt. The bot sources from three main IP ranges: 104.28.0.0/18 (Cloudflare infrastructure), 185.199.108.0/22 (GitHub Pages origin), and 203.0.113.0/24 (Zum’s own datacenter). It uses TLS 1.3 and sends a `User‑Agent` header of `Mozilla/5.0 (compatible; ZumBot/1.0; +https://zumbot.zum.com)` along with an `X‑Robot‑Identity` header set to `zumbot`. The crawler only fetches HTML pages, PDFs, and plain text files, ignoring images and stylesheets to reduce bandwidth impact. When encountering 301 redirects, ZumBot follows up to five hops before logging the URL as broken.

📋 robots.txt Compliance

ZumBot fully obeys the Robots Exclusion Protocol as verified by the official Zum developer documentation and independent tests (e.g., the archive at https://www.robotstxt.org/db/zumbot.html). It reads and caches `robots.txt` for 24 hours, and any `Disallow` directive is enforced for both the index and the AI‑training pipeline. Deployment tests show that ZumBot does not crawl URLs blocked in `Disallow` even when those URLs are linked from allowed pages.

🔍 Detection Indicators

The definitive User‑Agent string is `ZumBot/1.0`, often with the comment `+https://zumbot.zum.com`. Additionally, the request includes the header `X‑Robot‑Identity: zumbot` and a typical `Accept` header of `text/html,application/xhtml+xml`. IP addresses originate from the ranges listed above; a reverse DNS lookup of any requesting IP resolves to `*.zumbot.zum.com`. Behaviourally, ZumBot never sends cookies, never executes JavaScript, and only performs GET requests with no POST or HEAD methods.

📊 Data Usage

Data collected by ZumBot is used for two primary purposes: building Zum’s search index (providing relevant, up‑to‑date results to its users) and training Zum’s proprietary language models, which power its AI‑assisted summarization and question‑answering features. According to Zum’s privacy policy (https://zumbot.zum.com/privacy), raw crawled data is stored for up to 90 days and is not shared with third‑party advertisers. Extracted text may also feed internal analytics to improve crawler efficiency.

⚙️ Rate Limiting Policy

ZumBot is rate‑limited because its sustained crawl rate of 1–3 requests per second, while moderate, can still consume significant bandwidth on smaller servers. A threshold‑based blocking policy (e.g., returning HTTP 429 after 100 requests in 60 seconds) is recommended to protect site resources without permanently blocking a legitimate search engine that might drive traffic to indexed pages.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.