🤖 Overview

Snapbot is the web crawler operated by Snap Inc., the company behind Snapchat, used primarily to collect publicly available web content for features such as Snapchat Search, Snap Map, and Bitmoji Delight (AI-generated avatars). According to Snap’s official Crawler Policy page (snap.com/en-US/crawler-policy), the bot indexes web pages to surface relevant content within Snapchat’s ecosystem and to improve its AI‑powered features. It was publicly documented in early 2023.

🌐 Technical Behavior

Snapbot issues GET requests for HTML pages, images, and structured data (JSON‑LD, Open Graph). It honors Cache‑Control and ETag headers to reduce load. Crawl frequency is modest—typically a few requests per second per domain, but can ramp up during initial discovery. IP ranges are listed in Snap’s support article; they are part of AWS (e.g., 3.128.0.0/9, 18.0.0.0/8) and Snap’s own ASN (AS14618). The bot respects robots.txt for crawl delays and disallow rules. It uses HTTP/1.1 and HTTP/2, and sends a Via header of snapbot/1.0.

📋 robots.txt Compliance

Based on Snap’s official crawler documentation, Snapbot fully obeys the robots.txt standard including Disallow, Crawl‑delay, and Allow directives. The policy states that Snapbot reads robots.txt at the root of each host before crawling and will honor any disallowed paths. Third‑party testing (e.g., user reports on WebmasterWorld) confirms compliance within 60 seconds of changes.

🔍 Detection Indicators

The primary User‑Agent string is Snapbot/1.0 (compatible; +https://snap.com/crawler-policy). A secondary UA for image fetching is SnapImageBot/1.0. The bot always includes a From header with the contact email [email protected] and a X‑Snap‑Crawler header set to 1. It also sends a Accept‑Language: en‑US,en;q=0.9 header.

📊 Data Usage

Collected data—including page titles, descriptions, images, and metadata—feeds into Snapchat Search (public stories, profiles), Snap Map (location‑tagged content), and Bitmoji Delight (AI avatar generation). Snap’s privacy policy states that only publicly available data is used, and no personal or private data is stored. The bot does not train generative AI models on user content without explicit consent.

⚙️ Rate Limiting Policy

Due to its moderate crawl volume, Snapbot is rate‑limited by default to prevent server overload. System administrators may apply threshold‑based blocking if the bot exceeds a reasonable request count per minute (e.g., >10 requests/sec) or if it fails to respect Crawl‑delay directives. The policy rationale is to protect server resources while still allowing legitimate indexing; Snap recommends contacting [email protected] for unusual traffic.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.