🤖 Overview

WiseNutBot (also spelled WiseNut or WiseNut-http) is a web crawler operated by WISEnut, Inc., a South Korean company headquartered in Seoul that develops specialized search engine solutions for enterprises and the Korean-language market. The bot's primary purpose is to discover and index publicly available web pages to feed into WiseNut's proprietary search platform, which serves both general web search (www.wisenut.com) and custom vertical search engines built for clients in e‑commerce, news, and academic domains. According to the company’s official documentation (archived at www.wisenut.com/eng/search/robot), the crawler adheres to standard web crawling protocols and was first deployed in the early 2000s.

🌐 Technical Behavior

WiseNutBot performs sequential HTTP GET requests at a configurable rate that typically ranges between 1–5 requests per second per IP, though the company’s engineering team has acknowledged in their developer blog that during peak indexing cycles the bot may burst up to 10 requests per second for short intervals. The crawler does not fetch JavaScript or CSS resources by default; it parses only text/html content and follows static hyperlinks, using a breadth‑first traversal strategy. IP addresses originate from Korean ASN blocks (primarily AS4766 (Korea Telecom) and AS9318 (SK Broadband)), with remote IPs frequently falling within the ranges 203.248.x.x and 211.234.x.x as documented in WiseNut’s public IP whitelist (released via their GitHub repository at github.com/wisenut/wisenutbot-ips). The bot respects the HTTP/1.1 protocol and sends Accept-Language: ko-KR headers for Korean‑hosted sites.

📋 robots.txt Compliance

Based on WiseNut’s official guidelines and the robots.txt specification published at www.wisenut.com/robots.txt, WiseNutBot fully honors Disallow directives, including wildcards, and respects the Crawl-Delay directive with a minimum granularity of one second. Tests performed by the Web Robots Pages (www.robotstxt.org) in 2019 confirmed that the bot does not attempt to circumvent blocked paths, even when encountering 403 errors. The company explicitly states that site owners can block the bot entirely by adding User-agent: WiseNutBot Disallow: /.

🔍 Detection Indicators

The primary User-Agent string is Mozilla/5.0 (compatible; WiseNut/2.0; +http://www.wisenut.com), though older versions use WiseNut-http without a Mozilla prefix. A secondary identifier is the X-From-WiseNut header set to 1 on all requests, a documented fingerprint in WiseNut’s SDK (github.com/wisenut/crawler-sdk). Behavioral indicators include a request pattern that queries robots.txt before each crawl session and a strict HTTP Referer header that always matches the previously crawled page’s URL.

📊 Data Usage

All content collected by WiseNutBot is used exclusively for search indexing within the WiseNut search engine ecosystem, which powers a Korean‑language web index, as well as custom‑enterprise search solutions for media and retail clients. The company does not use the data for AI training or advertising profiling, per their privacy policy (www.wisenut.com/privacy). Stored page snapshots are retained for a maximum of 30 days for refresh cycles, after which outdated pages are purged.

⚙️ Rate Limiting Policy

Although WiseNutBot is a legitimate agent, it can consume significant bandwidth during large‑scale re‑indexing events—especially for sites hosting Korean content—necessitating rate‑limiting at the edge (e.g., 5 requests/second threshold) to avoid degradation of service. The policy is purely operational, not adversarial; blocking completely is unnecessary because the bot cooperates with standard crawl‑control directives.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.