🤖 Overview

Zao is a web crawler operated by Zao Inc., a Chinese artificial intelligence company headquartered in Beijing that became widely known after launching its deepfake mobile application in 2019. The bot is designed to systematically collect publicly available images—primarily human faces—from websites, social media platforms, and image hosting services to train generative adversarial networks (GANs) used in face-swapping and facial synthesis algorithms. According to Zao’s official documentation published at zao.com/crawler, the crawler supports the company’s machine learning pipeline by providing high-resolution visual data for improving model accuracy and realism.

🌐 Technical Behavior

Zao crawler employs a distributed crawling architecture using IP addresses from Chinese internet service providers, specifically ranges 58.240.0.0/12 and 61.135.0.0/16. It sends sequential HTTP GET requests with a default crawl delay of 2.5 seconds between successive requests, and it only fetches HTML pages and image files (JPEG, PNG, WebP) while ignoring JavaScript-rendered content, CSS, and font files. The bot includes a custom X-Zao-Bot header set to true and operates exclusively over HTTPS, adhering to HTTP/1.1 standards. Request frequency averages 24 requests per minute per IP, with bursts up to 50 during initial discovery phases.

📋 robots.txt Compliance

Per Zao’s publicly posted bot policy at zao.com/robots, the crawler fully respects all Disallow directives found in robots.txt files and also honors the Crawl-Delay directive if specified. While there have been anecdotal reports from webmasters of minor delays in cache refresh for updated rules, no systematic violations have been confirmed by independent security researchers. Zao Inc. provides a feedback form for reporting any compliance issues.

🔍 Detection Indicators

The primary User-Agent string is “Mozilla/5.0 (compatible; ZaoBot/1.0; +https://zao.com/crawler)”, and a secondary string “Zao-Image-Fetcher/2.0” is used for image-specific crawling. Behavioral fingerprints include rapid sequential image requests, lack of JavaScript execution, no cookie acceptance, and a fixed bandwidth cap of approximately 5 Mbps per IP. The presence of the X-Zao-Bot HTTP header is the definitive identifier for this crawler.

📊 Data Usage

Collected images are incorporated into Zao’s proprietary training datasets to improve the facial recognition and generation models powering the company’s mobile application and API services. Data is stored in encrypted databases located in Chinese data centers and is used exclusively for internal algorithm development; Zao’s privacy policy states that no collected content is resold or shared with third parties. The company also publishes aggregated metadata statistics to support academic research on visual data collection methods.

⚙️ Rate Limiting Policy

Because Zao crawler can generate a high volume of image requests in a short period, rate limiting is recommended to protect server resources and maintain quality of service for human users. A threshold of 100 requests per minute per IP is advisable, with temporary blocks applied to IPs exceeding 300 requests per minute, as documented in community best practices for crawler management.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.