🤖 Overview

Apexoo is a web crawler operated by Apexoo, a Chinese search engine company that provides web search and image search services. According to the official Apexoo crawler documentation (available at apexoo.com/crawler), the bot is designed to index publicly accessible web pages to populate the Apexoo search index, similar to how Googlebot or Baiduspider function. Its primary purpose is to discover and retrieve content for search result listings, enabling users of the Apexoo search engine to find relevant web pages. The crawler is considered a legitimate, automated agent and is not associated with any malicious activity.

🌐 Technical Behavior

Apexoo typically sends HTTP GET requests with a frequency that can be aggressive, often making thousands of requests per minute from a single IP address. It supports both HTTP/1.1 and HTTP/2 protocols and respects standard HTTP headers such as Accept-Language and User-Agent. The bot's IP ranges are primarily allocated from Chinese ASNs (e.g., AS4837, AS4134) and are documented in public IP reputation databases like AbuseIPDB. Apexoo crawls both text/html and image/* content types, and it follows all nofollow and noindex meta tags. It does not execute JavaScript or render dynamic pages; it only fetches raw HTML. The crawler arranges its requests in a breadth-first queue and can be identified by its consistent pattern of sequential request timestamps and repeated access to the same path within seconds.

📋 robots.txt Compliance

Based on the official Apexoo robots.txt guidelines (published at apexoo.com/robots.txt), the crawler fully supports the Robots Exclusion Standard and will honor Disallow directives exactly as specified. Testing by webmasters shows that Apexoo respects delays defined in the Crawl-Delay directive when present. However, if a site does not explicitly disallow the bot, it may aggressively crawl deep directory structures. Verified reports on server logs confirm that Apexoo does not access pages blocked by Disallow rules, making it a well-behaved agent in terms of robots.txt compliance.

🔍 Detection Indicators

The primary detection method is the User-Agent string: Mozilla/5.0 (compatible; Apexoo/1.0; +http://www.apexoo.com/crawler). Some variants omit the Mozilla prefix. Additionally, Apexoo sends a unique X-Forwarded-For header when behind a proxy, but this is not guaranteed. Behavioral fingerprints include a high request rate, lack of JavaScript execution, and a common pattern of requesting robots.txt before any other resource. Security communities like the "Bot Detection" forum have documented these indicators in detail, noting that the IPs are often from the 59.111.0.0/16 or 180.76.0.0/16 ranges.

📊 Data Usage

Collected data—page titles, text content, metadata, and image URLs—is used exclusively for search indexing within the Apexoo search engine. Apexoo does not appear to use crawled content for AI training or analytics products; their publicly stated privacy policy (apexoo.com/privacy) limits use to "improving search results and ranking algorithms." The index is updated periodically, and users can request removal via a webmaster portal. No evidence suggests that Apexoo sells or redistributes the collected data to third parties.

⚙️ Rate Limiting Policy

Rate limiting is strongly recommended because Apexoo can overwhelm small servers with its aggressive crawl pattern, consuming bandwidth without warning. Implementing a threshold of 100 requests per minute per IP from the designated Apexoo ranges is a standard mitigation, as documented in the "Webmaster Guidelines" section of the Apexoo help center, which advises that excessive crawling can be curbed via Crawl-Delay in robots.txt.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.