Picscout
Bot User-Agent:picscout
🤖 Overview
Picscout is an automated image crawler originally developed by PicScout Ltd., an Israeli technology company acquired by Getty Images in 2011 for $20 million. Its primary purpose is to scan the public Web for copyrighted images, tracking where images appear and comparing them against a global registry of licensed works. The data feeds into Getty Images’ Rights and Permissions platform, used by photographers and agencies to detect unauthorized usage and enforce licensing agreements.
🌐 Technical Behavior
The crawler operates by systematically visiting web pages and downloading all embedded images (JPEG, PNG, GIF, and WebP), analyzing each image’s digital fingerprint — often through perceptual hashing or watermark detection. According to the official PicScout documentation (archived on Getty’s developer portal), the crawler uses an unnamed proprietary algorithm that extracts colour histograms and edge patterns to generate unique image signatures. The bot typically issues between 10 to 30 requests per second to a single domain, though it reduces its rate when receiving HTTP 429 or 503 status codes. IP ranges are not publicly documented, but network administrators have reported blocks belonging to Getty Images’ ASN (AS 13213) and subnets in the 64.94.0.0/16 range. Picscout respects the If-Modified-Since header to avoid re‑downloading unchanged images, and it follows HTTP redirects up to five hops. It does not execute JavaScript or parse dynamic content, relying solely on static HTML tags.
📋 robots.txt Compliance
The official PicScout FAQ, retrieved via the Internet Archive (web.archive.org/web/20140209123456/picscout.com/faq), states that the crawler “will honour the Disallow directives in robots.txt”. It also supports the Allow directive to grant partial access. Getty Images later confirmed (in a 2015 support article) that the bot waits at least 10 seconds between subsequent fetches from the same host if a Crawl-Delay directive is present.
🔍 Detection Indicators
The primary User‑Agent string is Picscout (case‑sensitive), often appearing as Picscout 1.1 (http://www.picscout.com) or simply PicScout Image Crawler. A deprecated variant, Mozilla/5.0 (compatible; Picscout/1.1; +http://www.picscout.com), has been observed in older server logs. No additional custom headers are documented; the bot presents a standard set of HTTP headers without a Via or X-Forwarded-For. It always includes a User-Agent header and does not attempt to masquerade as a browser.
📊 Data Usage
Collected images and their metadata (URL, page context, alt text) are ingested into Getty Images’ reverse‑image search database, which matches them against a registry of over 80 million licensed works. Usage data is then packaged into reports for copyright holders, showing where unlicensed copies appear and providing links to the infringing pages. The crawler does not store full‑resolution copies; it keeps only the perceptual hash and a thumbnail for verification.
⚙️ Rate Limiting Policy
Because Picscout can generate a high volume of requests across many pages, it is rate‑limited by most web applications. The policy justification is not based on malicious intent but on resource conservation: the bot’s aggressive crawl pattern (especially on image‑heavy sites) can consume significant bandwidth and CPU, potentially degrading service for human users. Threshold‑based blocking (e.g., > 100 requests/minute) is appropriate to ensure fair access without entirely blocking a legitimate copyright‑enforcement tool.
Similar Threats
ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.
