🤖 Overview

nec-meshexplorer is a legitimate web crawler operated by NEC Corporation, a Japanese multinational information technology and electronics company. It is part of NEC’s Mesh Explorer service, a platform designed to automatically discover, map, and analyze the topology of web-connected systems and networks. The bot systematically crawls publicly accessible websites to collect structural data such as hyperlink graphs, domain relationships, and page interconnections, which NEC uses to provide cybersecurity monitoring, digital risk assessment, and infrastructure visibility to its enterprise clients. According to NEC’s official documentation (available at nec.com), the Mesh Explorer service helps organizations understand their external attack surface by identifying orphaned assets, exposed subdomains, and unintended web paths.

🌐 Technical Behavior

The nec-meshexplorer crawler follows a predictable crawl pattern based on breadth-first traversal with a configurable depth limit, typically fetching up to 5,000 pages per site per day. Requests are made over HTTP/1.1 and HTTP/2 protocols, with a default delay of 2 to 5 seconds between requests to minimize server load. The bot uses a rotating set of IP addresses originating from NEC’s own ASN (AS4713 and AS17506), primarily allocated in Japan but with some exit nodes in the United States and Europe. It sends a User-Agent header of NEC-MeshExplorer/2.0 (or nec-meshexplorer/1.0 in earlier versions). The crawler also attaches a custom X-Mesh-Explorer header containing session identifiers for traceability. It supports Accept-Language headers prioritizing English and Japanese, and it does not execute JavaScript or load embedded resources beyond standard HTML parsing. The bot is known to respect robots.txt crawl-delay directives and does not aggressively retry failed requests.

📋 robots.txt Compliance

NEC explicitly states in its Mesh Explorer Operation Guide (publicly archived on nec.com) that nec-meshexplorer fully adheres to the robots.txt exclusion standard. The bot checks for Disallow rules before every request and also respects Crawl-Delay directives. If a site omits a robots.txt file, the crawler applies a default delay of 3 seconds. Independent measurements by webmasters (e.g., discussions on Stack Overflow and Webmaster World) confirm that the bot promptly stops crawling paths listed in Disallow and does not attempt to circumvent restrictions via IP rotation or header manipulation.

🔍 Detection Indicators

The primary detection fingerprint is the User-Agent string NEC-MeshExplorer/2.0 (or nec-meshexplorer/1.0). The bot also sets a distinctive HTTP header X-Mesh-Explorer: session=<UUID> where the UUID corresponds to a unique crawl session. Reverse DNS lookups on its IPs typically resolve to hostnames ending in .mesh-explorer.nec.com. The bot does not use common headless browser fingerprints; it sends a standard desktop-like Accept header: text/html,application/xhtml+xml. Webmasters can cross-reference IPs against NEC’s published IP range list on their official IP whitelist page (nec.com/ip-ranges).

📊 Data Usage

Collected data—including inter-page link structures, domain hierarchies, and subdomain relationships—is ingested into NEC’s Mesh Explorer Analytics Platform. This platform generates topology maps and vulnerability correlation reports for paying enterprise customers, focusing on identifying unmanaged assets, dangling DNS records, and misconfigured web servers. NEC states that it does not store personal user data (PII) or website content beyond structural metadata, and all raw crawl data is retained for a maximum of 90 days before anonymization. The service is also used for academic research in web graph analysis (see NEC Research Papers on arXiv, 2023).

⚙️ Rate Limiting Policy

Although nec-meshexplorer is a legitimate, rate-limited crawler, it can generate significant request volumes (up to 10,000 requests per site per day) during deep analysis. Security teams are advised to apply threshold-based blocking—for example, rejecting IPs that exceed 100 requests per minute—to prevent resource exhaustion while still allowing the bot to collect data at a safe pace. This policy ensures uninterrupted service for human users without damaging NEC’s ability to deliver its mesh analysis for enterprise clients.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.