🤖 Overview

360Spider is a web crawler operated by Qihoo 360, a Chinese cybersecurity and search engine company headquartered in Beijing. It was introduced to power the 360 Search engine (so.360.cn) and related products, collecting publicly available web content for indexing and ranking. Qihoo 360 is publicly traded (NYSE: QIHU until 2016) and its crawler is a legitimate search engine bot.

🌐 Technical Behavior

360Spider crawls the web using HTTP/1.1 requests with a configurable crawl delay, typically between 10 and 30 seconds per host. It respects the robots.txt file but has been observed to make rapid consecutive requests if no delay is specified. Its IP ranges are concentrated in China, primarily from the AS4808 (China Unicom) and AS4837 (China Telecom) autonomous systems, with addresses like 123.125.71.xx and 180.76.15.xx. The crawler uses HTTPS for secure sites and follows standard HTTP redirects. It indexes both static HTML and JavaScript-rendered content through its own rendering engine. Official documentation from 360.cn (Chinese) states it operates 24/7 with a maximum of 50 concurrent connections per domain.

📋 robots.txt Compliance

360Spider fully honors Disallow directives in robots.txt as documented by Qihoo 360’s official webmaster guidelines (so.360.cn/public/robots.html). It also respects Crawl-Delay and User-Agent‑specific rules. In practice, compliance is reliable, though some webmasters report occasional rate bursts during initial indexing.

🔍 Detection Indicators

The primary User-Agent string is Mozilla/5.0 (compatible; 360Spider/1.3; +http://so.360.cn/spider.html). Secondary variants include 360Spider/1.2 and 360Spider/1.0. Behavioral fingerprints include a high request rate for favicon.ico and robots.txt, and a ​Via header sometimes containing “360Spider”. The bot identifies itself in the User-Agent header and does not use obfuscation.

📊 Data Usage

Collected data is exclusively used for 360 Search indexing and ranking, as well as for Qihoo 360’s security products (e.g., web page reputation analysis). No evidence indicates the data is sold to third parties or used for AI training. The official policy states that data is retained only as long as needed for search freshness.

⚙️ Rate Limiting Policy

Rate limiting 360Spider is recommended when its crawl frequency exceeds 10 requests per second per IP, as the bot can consume significant bandwidth during initial site scans. A threshold-based blocking approach (e.g., 5 requests per second for 60 seconds) protects server resources without permanently denying legitimate indexing.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.