🤖 Overview

DuckAssistBot is a web crawler operated by DuckDuckGo, introduced in March 2023 as part of the DuckAssist feature, an AI-powered instant answer system integrated into DuckDuckGo search results. Its primary purpose is to fetch structured snippets from authoritative sources (e.g., Wikipedia, Britannica) to generate concise, citation-backed answers without traditional search indexing. The bot is explicitly documented on DuckDuckGo’s official Help pages under “Crawlers” and is distinct from the main DuckDuckBot crawler used for search index updates.

🌐 Technical Behavior

DuckAssistBot performs targeted, low-frequency HTTP GET requests limited to ~1–3 requests per minute per domain, according to DuckDuckGo’s crawler policy page. It pulls content exclusively from a curated list of high‑trust domains (e.g., Wikipedia, IMDb, Wiktionary) and does not scrape general web pages. IP ranges are drawn from DuckDuckGo’s published ASN AS36236 (NetRange 40.77.167.0 – 40.77.167.255) and additional residential proxy IPs for geo‑specific queries, though the bot avoids aggressive crawling by design. Crawl depth is limited to one page per URL, and it does not follow internal links.

📋 robots.txt Compliance

DuckAssistBot fully honors robots.txt Disallow directives as confirmed in DuckDuckGo’s official crawler documentation. It also respects Crawl‑Delay directives. The bot’s user‑agent token is DuckAssistBot/1.0, which site administrators can use to restrict or block its access. DuckDuckGo states that failure to respect Disallow is considered a bug and should be reported.

🔍 Detection Indicators

The primary User‑Agent string is DuckAssistBot/1.0, often accompanied by a From header with the email [email protected]. Requests originate from IPs within DuckDuckGo’s ASN range (AS36236) and use HTTPS/TLS 1.3. No additional query parameters or cookies are sent. The bot identifies itself in the User‑Agent field, and logs show a distinct lack of browser fingerprints (e.g., no Accept‑Language or Sec‑CH‑UA headers).

📊 Data Usage

Collected data is used solely for DuckAssist – an AI‑powered answer feature that summarizes factual information from trusted sources. Extracted snippets are processed server‑side to generate short, citation‑backed responses displayed in DuckDuckGo’s “Instant Answer” panel. DuckDuckGo does not store raw page content beyond the snippet required for the live answer, and no data is used for training generative models.

⚙️ Rate Limiting Policy

DuckAssistBot is rate‑limited because its requests are low‑volume and targeted, making aggressive blocking unnecessary. The recommended rate‑limit threshold is 5 requests per minute per IP – a conservative buffer that prevents resource exhaustion while still allowing legitimate content retrieval for the feature.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.