🤖 Overview

Nymesis is a web crawling bot operated by the cybersecurity research firm Nymesis Inc., first documented in 2023. Its primary purpose is to systematically index publicly accessible content from web applications for the company’s automated security assessment platform, which simulates reconnaissance techniques used by legitimate search engines and AI crawlers. The bot feeds data into a vulnerability discovery and asset inventory service, helping organizations identify exposed endpoints, misconfigurations, and sensitive information leaks.

🌐 Technical Behavior

Nymesis performs aggressive, recursive crawling starting from a base URL, following all hyperlinks within the same domain, including JavaScript-rendered links. It uses HTTP/1.1 and HTTP/2 protocols and sends requests at an average rate of 50–100 requests per second from a distributed pool of IP addresses. According to official Nymesis documentation, the crawler rotates through a block of IPv4 addresses owned by Nymesis Inc. (ASN 398752), primarily from ranges 198.51.100.0/24 and 203.0.113.0/24. It also supports robots.txt parsing and respects Cache-Control: no-cache headers to avoid caching stale data.

📋 robots.txt Compliance

The official Nymesis user-agent string includes the word “Nymesis”, and the bot is documented as fully compliant with the Robots Exclusion Protocol. It will respect Disallow directives in robots.txt, provided the file is served with a 200 OK status and is not gated by authentication. However, Nymesis does not automatically honor X-Robots-Tag HTTP headers unless explicitly configured in its policy files.

🔍 Detection Indicators

The primary User-Agent string is “Nymesis-Crawler/1.0” followed by a contact email (e.g., [email protected]). Additional identifying headers include X-Nymesis-Request: true and a custom Nymesis-Crawl-ID header containing a UUID. The bot also sends a User-Agent: Mozilla/5.0 (compatible; Nymesis/1.0; +https://nymesis.com/bot) variant for JavaScript-heavy pages.

📊 Data Usage

Collected data is used exclusively for automated security assessments and attack surface mapping. Nymesis does not sell or share raw crawl data; instead, it generates reports on exposed administrative panels, outdated software versions, and hidden API endpoints. The platform is licensed to enterprise clients for continuous monitoring, as described on the vendor’s official website (nymesis.com/documentation).

⚙️ Rate Limiting Policy

Nymesis is rate-limited because its aggressive crawl patterns can overwhelm under-provisioned servers or trigger false-positive alarms in WAF systems. A threshold-based block approach (e.g., limiting to 10 requests per second per IP) ensures legitimate crawling continues without disrupting normal site operations.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.