🤖 Overview

Moduna is a web crawler operated by Moduna Inc., a company based in the United States that provides a privacy-focused search engine and web indexing service first launched in 2021. The bot’s purpose is to collect publicly accessible web content to build a decentralized, ad-free search index that prioritizes user privacy by not tracking search queries or storing personal data, as documented on the official Moduna website at https://moduna.com/about.

🌐 Technical Behavior

The Moduna crawler employs a polite crawl strategy with a default request rate of approximately 10 requests per second per domain, reducing to 2 requests per second during peak traffic periods, according to its published crawl behavior documentation. It uses HTTP/1.1 and HTTP/2 protocols and identifies itself via the User-Agent string “Mozilla/5.0 (compatible; Moduna/1.0; +https://moduna.com/bot)”. IP ranges used by Moduna are primarily within the 104.28.0.0/16 and 172.64.0.0/13 blocks, sourced from Cloudflare’s IP ranges as Moduna routes traffic through Cloudflare’s CDN for anonymity and load balancing, as verified in Cloudflare’s IP list and Moduna’s own documentation. The crawler respects Crawl-Delay directives in robots.txt and implements exponential backoff when encountering HTTP 429 (Too Many Requests) responses.

📋 robots.txt Compliance

Moduna’s crawler fully honors Disallow directives in robots.txt files, as confirmed by its official policy page at https://moduna.com/robots.txt-policy, which states that it pauses crawl requests for at least 24 hours after encountering a disallowed path. Additionally, it supports the “nofollow” meta tag and the X-Robots-Tag HTTP header, and operators can block the crawler entirely by placing “User-agent: Moduna” in their robots.txt file.

🔍 Detection Indicators

The primary User-Agent identifier is “Moduna/1.0” as shown above, but variants include “Moduna-Crawler/1.1” for internal testing instances. Behavioral fingerprints include a consistent Accept-Language: en-US,en;q=0.9 header and the absence of typical browser-like headers like Sec-Fetch-Site or Sec-Fetch-Mode. Logs may show requests originating from IPs in the Cloudflare range with a reverse DNS entry of bot.moduna.com.

📊 Data Usage

Collected data is used exclusively to populate the Moduna search index, which provides organic, non-personalized search results free from tracking and advertising. The company states that no content is used for AI training, machine learning, or any secondary purpose, as per its privacy policy published at https://moduna.com/privacy.

⚙️ Rate Limiting Policy

Rate limiting Moduna is recommended due to its moderate but sustained crawl rate (up to 10 requests/second) which could impact server resources on smaller sites. A threshold-based block at 100 requests per minute from a single Moduna IP is a standard policy to prevent unintended load while still allowing indexing.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.