🤖 Overview

nabot is a legitimate web crawler operated by Naver Corporation, the South Korean internet conglomerate behind the Naver search engine. Its primary purpose is to discover, fetch, and index publicly accessible web content for Naver’s search results, news aggregation, and knowledge graph services. First documented in Naver’s official crawler guidelines, nabot has been active since the early 2010s and is distinct from other Naver bots such as “NaverBot” or “Yeti”. It is part of Naver’s infrastructure for maintaining a comprehensive, Korean-language-dominant search index.

🌐 Technical Behavior

nabot employs a standard HTTP/1.1 request pattern with a configurable crawl delay that defaults to several seconds between requests to a single host. According to Naver’s published technical documentation, it uses IP addresses primarily from the 121.78.0.0/16 and 61.100.0.0/16 ranges, allocated to Naver’s datacenters in South Korea. The crawler sends If-Modified-Since and ETag headers to reduce bandwidth usage and respects the X-Robots-Tag HTTP header. It does not follow JavaScript redirects by default, but can execute basic HTTP redirects (301, 302). The bot’s request frequency is influenced by the Crawl-Delay directive in robots.txt, although Naver recommends a default of 5 seconds per host. No evidence of parallel burst requests has been reported; instead, nabot uses sequential, single-threaded crawling per origin.

📋 robots.txt Compliance

Naver explicitly states that nabot honors Disallow directives in robots.txt, as detailed in the official Naver Search Advisor (searchadvisor.naver.com). The crawler also respects the Allow directive and supports User-agent: nabot lines. There are no documented cases of nabot ignoring robots.txt; compliance is enforced by Naver’s crawl policy. The bot additionally recognizes the Sitemap directive to discover priority pages.

🔍 Detection Indicators

The primary User-Agent string for this bot is "nabot/1.0" or "nabot". A secondary variant, "Mozilla/5.0 (compatible; nabot/1.0; +https://help.naver.com/robots/)", is also observed. The bot includes a From header with a contact email (e.g., [email protected]) and a Referer header that often defaults to an empty string. Reverse DNS lookups on its IPs resolve to subdomains like crawl.*.navercorp.com. Behavioral fingerprints include a consistent 5-10 second gap between requests and a lack of JavaScript rendering.

📊 Data Usage

Collected data—including page text, metadata, and structure—is used exclusively to populate and refresh Naver’s search index. Naver also employs the crawled content to train its internal NLP models for Korean-language search ranking and for its AI-powered assistant (Clova). The data is not sold to third parties; it remains within Naver’s ecosystem for improving search relevance and user experience.

⚙️ Rate Limiting Policy

Although nabot is a legitimate commercial crawler, it is rate-limited by most web administrators because its steady crawl patterns can consume significant bandwidth over extended periods. The policy rationale is to prevent any single crawler from degrading site performance for human users; threshold-based blocking (e.g., after 100 requests per minute) is recommended as a balancing measure between accessibility and server resource preservation.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.