🤖 Overview

The Aport crawler is operated by the Russian search engine and price comparison service Aport (formerly part of the Rambler group, now independently maintained). Its primary purpose is to index web pages, images, and product listings for the Aport search engine and shopping aggregator, which launched in 1996 and remains active in Russia and CIS countries. The bot collects publicly accessible content to build Aport’s search index and e‑commerce database.

🌐 Technical Behavior

Technically, the Aport bot performs HTTP GET requests with a configurable crawl delay, typically between 2 and 10 seconds per request, though it can accelerate when no rate‑limiting is enforced. Its requests often include the Accept‑Language: ru header and originate from IP ranges registered to Russian hosting providers such as Selectel and Rostelecom. According to official documentation on aport.ru, the bot supports HTTP/1.1 and HTTPS, and it follows standard redirection status codes (301, 302). It does not parse JavaScript or execute client‑side scripts, focusing instead on static HTML and structured data (e.g., schema.org markup).

📋 robots.txt Compliance

The Aport crawler officially honors the robots.txt standard, including both Disallow and Crawl‑delay directives. Verified by tests and documented on the Aport webmaster portal, it will obey a Crawl‑delay: 5 instruction. However, if no delay is set, the bot may send requests at a rate of up to 10 per second, which can be aggressive for small sites. It does not ignore rules unless misconfigured.

🔍 Detection Indicators

Detection is straightforward: the User‑Agent string is either Aport or Aport/1.0 (e.g., Mozilla/4.0 (compatible; Aport)). The bot may also include a From header with a contact email (e.g., [email protected]). Behavioral fingerprints include sequential IP access from the same /24 subnet and a high proportion of requests for .html and .php pages.

📊 Data Usage

Collected data is used exclusively for Aport’s search index and shopping price comparison service. The bot stores page titles, meta descriptions, product prices, and availability information. It does not train AI models or generate derivative content; instead, results are served directly in Aport’s SERP and price‑comparison pages. No personal data is retained beyond what is publicly visible on the indexed page.

⚙️ Rate Limiting Policy

Rate limiting is applied because the Aport bot can generate high request volumes (up to 10 req/s) when no delay is specified, potentially degrading server performance or triggering false‑positive DDoS alarms. A threshold of 5 requests per second per IP is recommended to balance crawl completion speed with server resource protection.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.