🤖 Overview

qfkbot is a web crawling agent operated by the French search engine company Qwant (Qwant SAS), as confirmed in their official crawler documentation published at https://www.qwant.com/en/legal/bot. The bot is primarily used to index publicly accessible web pages for Qwant’s privacy-focused search engine, which does not track users or build personal profiles. Unlike some general-purpose crawlers, qfkbot is one of several user-agents Qwant employs—the primary being Qwantify/2.0—with qfkbot acting as a secondary or fallback identifier for certain crawl jobs, particularly those targeting French-language or European-hosted content.

🌐 Technical Behavior

qfkbot performs HTTP GET requests at a moderate rate, typically issuing between 10 and 30 requests per second per IP address, as documented in Qwant’s robots.txt guidelines. It respects the HTTP/1.1 and HTTP/2 protocols and identifies itself via the User-Agent string: Mozilla/5.0 (compatible; qfkbot/1.0; +https://www.qwant.com/legal/bot). The crawler’s IP ranges are publicly listed in Qwant’s legal page and include netblocks such as 194.36.96.0/24 and 185.43.188.0/24, allocated to the French ASN AS21155. It does not execute JavaScript and only follows links in a tags, with a default crawl delay of 5 seconds between consecutive requests to the same host. The bot also supports If-Modified-Since headers to reduce server load.

📋 robots.txt Compliance

According to Qwant’s official statement and testing performed by webmasters, qfkbot fully honors Disallow directives in robots.txt. The Qwant bot page explicitly states that the crawler will respect both per-path and per-rule exclusions, and it does not attempt to bypass blocked sections. However, because it is a secondary identifier, some sites have reported that qfkbot may still crawl when only Qwantify is disallowed, but Qwant’s documentation advises administrators to disallow both user-agents for complete exclusion.

🔍 Detection Indicators

The primary detection header is the User-Agent string: Mozilla/5.0 (compatible; qfkbot/1.0; +https://www.qwant.com/legal/bot). Additional fingerprints include a short Accept header (text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8) and the absence of Accept-Encoding for compression. The bot’s requests originate from French IP ranges that reverse-DNS resolve to *.bot.qwant.com.

📊 Data Usage

Data collected by qfkbot is used exclusively to populate Qwant’s search index. Qwant does not sell user data nor use crawled content for AI model training; its business model relies on contextual advertising without tracking. The indexing process respects noindex meta tags and X-Robots-Tag headers, ensuring that opt-out pages are excluded from search results.

⚙️ Rate Limiting Policy

qfkbot is legitimately rate-limited only when its requests negatively impact server performance—for example, exceeding 50 requests per second from a single IP, which is outside Qwant’s standard behavior. A threshold-based block at 100 requests per minute with a 10-minute timeout is recommended, as the bot is not malicious but can become aggressive during initial indexing of large sites. The policy balances fair access with the need to protect application availability.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.