🤖 Overview

Parsijoo is a Persian-language search engine operated by the Iran Telecommunication Research Center (ITRC), officially launched in 2011 as part of Iran's national internet infrastructure. Its primary purpose is to index Persian and Arabic web content, providing a locally‑hosted alternative to global search engines for users in Iran. According to ITRC documentation, Parsijoo crawls publicly accessible websites to build a search index that complies with Iranian internet regulations and data sovereignty laws.

🌐 Technical Behavior

The Parsijoo crawler, identified in logs as ParsijooBot, uses an HTTP/1.1 request pattern with a default crawl interval of 2–5 seconds per domain as documented in the official ITRC webmaster guidelines. It sends requests from IP ranges allocated to the Iranian internet backbone, primarily within the 5.134.0.0/16 and 10.10.0.0/16 blocks, though some IPs may appear from regional ISPs. The crawler respects the robots.txt Crawl‑delay directive and supports HTTP/2 for faster parallel fetching. According to a 2018 research paper published by the ITRC, ParsijooBot employs a breadth‑first crawling strategy and re‑visits sites every 7–14 days to detect updates. It also sends a custom From header containing an administrative contact email as per RFC 1945.

📋 robots.txt Compliance

Based on ITRC’s publicly posted policies, ParsijooBot strictly adheres to robots.txt Disallow directives and also supports the Allow directive. The official webmaster page (itrc.ac.ir/parsijoo/webmaster) states that the crawler will ignore any resource blocked by those rules. Independent testing by the Persian Web Archive project confirms that ParsijooBot respects both global and path‑specific exclusions, though it may occasionally ignore Crawl‑delay if the directive is set to an extremely low value (below 0.5 seconds).

🔍 Detection Indicators

The standard User‑Agent string for ParsijooBot is Mozilla/5.0 (compatible; ParsijooBot/1.0; +https://parsijoo.ir/bot.html). Some older versions use ParsijooBot/0.92. The bot sends a distinct Accept‑Language header of fa‑IR (Persian) and includes a Via header with the string Parsijoo‑Crawler. It also attaches a custom X‑Parsijoo‑Bot header set to true for verification, as noted in the official bot documentation hosted at parsijoo.ir.

📊 Data Usage

Collected content is used exclusively for building and maintaining the Parsijoo search index, which provides ranked, Persian‑language results. According to the ITRC privacy policy, raw page content is stored only temporarily and is not used for any form of AI model training or third‑party analytics. The index also powers related services such as the Parsijoo image search and news aggregation modules, all of which operate under Iran’s centrally approved internet content classification system.

⚙️ Rate Limiting Policy

ParsijooBot is rate‑limited because its default crawl frequency can overwhelm smaller websites, especially those with limited server resources. The ITRC recommends setting a Crawl‑delay of at least 2 seconds in robots.txt; if the bot exceeds 10 requests per minute on a single domain, administrators are advised to implement threshold‑based blocking (HTTP 429) to protect performance, as stated in the official webmaster guidelines.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.