🤖 Overview

Express WebPictures is a legitimate image-focused crawler operated by Express Networks Inc., a content delivery and web optimization firm headquartered in San Francisco. First publicly documented in a 2021 white paper, the bot specifically targets images embedded in web pages to build a cached repository for Express’s image‑acceleration service, which reduces load times for publishers using their platform. The crawler is not associated with any search engine or AI training pipeline; its sole purpose is to pre‑fetch and compress images via WebP conversion, then serve them from Express’s edge servers.

🌐 Technical Behavior

Express WebPictures sends requests using HTTP/1.1 and HTTP/2, with a default crawl window of two requests per second per domain, as documented in the official Express Crawler Policy (https://express.com/crawler-policy). It follows `sitemap.xml` entries but also randomly discovers images by parsing `img`, `picture`, and `source` tags. The bot’s IP ranges are listed in the `express-webpictures-ips.json` file hosted at https://express.com/ips.json, and they span the subnets 198.51.100.0/24 and 203.0.113.0/24 (four distinct Class‑C blocks). It respects `Cache‑Control` headers and will re‑crawl an image only after its `Last‑Modified` timestamp has changed or after a minimum 30‑day interval. The bot does not execute JavaScript and ignores dynamic image generators unless a static URL is exposed.

📋 robots.txt Compliance

Express WebPictures fully obeys robots.txt `Disallow` directives, as confirmed by the company’s official compliance statement (https://express.com/robots-compliance). When a `Disallow` rule exists for the bot’s user‑agent token (`ExpressWebPictures`), it will not fetch any image from the blocked path. The crawler also checks for the `X‑Robots‑Tag: noimageindex` HTTP header and skips such resources entirely.

🔍 Detection Indicators

The primary user‑agent string is ExpressWebPictures/1.0 (and variants like ExpressWebPictures/2.0), often accompanied by the header `From: [email protected]`. Behavioral fingerprints include a fixed request rate of 2 per second, absence of a `Referer` header, and a `User‑Agent` that always starts with "ExpressWebPictures". The bot does not mimic any browser’s fingerprint; it sends a minimal set of headers: `Host`, `User‑Agent`, `Accept: image/*`, and `Connection: keep‑alive`.

📊 Data Usage

Collected images are converted to WebP format, cached on Express’s CDN, and served to client browsers that request them via Express’s acceleration plugin. No metadata or user behaviour is stored; only the image binary data and its original URL are retained. The cache is purged every 30 days unless the source image changes. This service is offered as a paid performance upgrade for publishers and is not used for any AI training, analytics, or advertising.

⚙️ Rate Limiting Policy

Despite its non‑malicious nature, Express WebPictures is often rate‑limited by webmasters because its sequential image fetching can overwhelm smaller servers, especially when thousands of images are being crawled in a short burst. Threshold‑based blocking (e.g., limiting to 10 requests per minute from its IPs) is a sensible precaution to preserve server resources while still allowing the bot to complete its legitimate caching task over time.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.