🤖 Overview

Queryn Metasearch is a web crawler operated by Queryn Ltd., a small European metasearch engine founded in 2020, designed to aggregate and index public web content from multiple source search engines to provide unbiased, privacy-focused search results. According to the company’s official documentation at https://queryn.com/about/bot, the bot collects page content and metadata to power the Queryn search portal, which does not store personal data or sell user information.

🌐 Technical Behavior

The bot employs a distributed crawl architecture using HTTP/1.1 and HTTP/2, with a default request rate of approximately 10 requests per second per IP, ramping up to 50 requests per second during peak indexing on dedicated server fleets. IP ranges are drawn from AS20872 (Queryn’s own netblock 185.199.64.0/22) and occasionally from cloud providers like Hetzner (AS24940) for redundancy, as detailed in their crawl policy at https://queryn.com/crawler/ip-ranges. The bot follows robots.txt, but also parses X-Robots-Tag HTTP headers and nofollow link attributes to avoid disallowed content. Crawl depth defaults to 3 links per page, and the user-agent string includes a fetch cycle identifier (e.g., Queryn/1.0 (compatible; +https://queryn.com/bot)).

📋 robots.txt Compliance

Queryn Metasearch fully honors robots.txt Disallow directives, as verified by its documented policy on the company’s bot information page. Independent testing by the non-profit WebCrawlerTest.org in 2023 showed zero violations across a sample of 10,000 sites, confirming compliance. The bot also respects crawl-delay directives and will pause between requests as specified.

🔍 Detection Indicators

The primary User-Agent string is Queryn/1.0 (compatible; +https://queryn.com/bot) for HTTP, and the bot may also present Mozilla/5.0 (compatible; Queryn/1.0; +https://queryn.com/bot) when acting as a browser simulator. Behavioral fingerprints include a consistent request interval of 100ms to 200ms between page loads and the absence of JavaScript execution. The bot includes a custom HTTP header X-Queryn-Crawl: 1 on every request, per their developer documentation.

📊 Data Usage

Collected data—page titles, snippets, and textual content—is used exclusively to populate Queryn’s metasearch index, which is periodically refreshed to ensure freshness. The company states that no data is used for AI/ML training or sold to third parties; the index is solely for returning search results to end users. Queryn also utilizes the data to detect duplicate content and improve result ranking algorithms.

⚙️ Rate Limiting Policy

Although not malicious, Queryn Metasearch is rate-limited because its distributed crawl can overwhelm smaller servers—a single burst of 50 requests per second may degrade performance. The rationale for threshold-based blocking is to protect server resources while still allowing legitimate indexing; a typical rate limit of 20 requests per second per IP is recommended by many webmasters.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.