image-fetcher
image-fetcher is a legitimate web crawler operated by DuckDuckGo, used to fetch and cache images for display in DuckDuckGo image search results. First documented in 2019, its sole purpose is to retrieve image data (JPEG, PNG, GIF, WebP) from public web pages to build the search engine’s image index.
The image-fetcher bot sends HTTP GET requests with a variable crawl rate, typically 1–5 requests per second per IP, but can burst up to 20 requests in short intervals during initial index updates. It uses IPv4 ranges published in DuckDuckGo’s official IP list (e.g., 50.18.x.x and 54.193.x.x, as confirmed in DuckDuckGo’s crawler documentation at duckduckgo.com/duckduckgo-help-pages/results/sources/). The bot fetches only images, not HTML pages, and does not execute JavaScript or follow links. It requests images with Accept: image/* and ignores Content-Type headers other than image types. The User-Agent string includes the bot’s version and a contact URL for feedback (see official DuckDuckGo crawler page).
According to DuckDuckGo’s official guide, image-fetcher fully respects robots.txt Disallow directives, including the User-agent: DuckDuckGo-Image-Fetcher line. It also obeys Crawl-Delay instructions if specified, though DuckDuckGo recommends site owners use Disallow for images they don’t want indexed.
The User-Agent string is DuckDuckGo-Image-Fetcher/1.0 (+https://duckduckgo.com/duckduckgo-help-pages/results/sources/). The bot also sends the header From: [email protected] (a generic DuckDuckGo contact) and Accept-Encoding: gzip. Reverse DNS lookups for its IPs resolve to *.duckduckgo.com.
Collected images are stored temporarily in DuckDuckGo’s image cache and used solely to generate thumbnail previews in search results. The images are not used for AI training or any other product; they are discarded after the cache expires (typically 7–14 days).
image-fetcher is rate-limited because high volumes of requests during initial index crawls can strain server resources. A threshold of 20 requests per 10 seconds per IP is recommended to prevent overloading while still allowing the bot to update the image index.
Similar Threats
⚠️
Unwanted bots inflate your analytics, drain server resources, and slow down real users. Check if your site is affected — completely free.
Check My Site for FreeFree to start · Cancel anytime
ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.
Stay up to date with the latest from Boteraser.
We use cookies to improve your experience on our site. By using our site, you consent to cookies.
Manage your cookie preferences below:
Essential cookies enable basic functions and are necessary for the proper function of the website.
CloudFlare provides web performance and security solutions, enhancing site speed and protecting against threats.
Service URL: developers.cloudflare.com (opens in a new window)
These cookies are needed for adding comments on this website.
These cookies are used for managing login functionality on this website.
Statistics cookies collect information anonymously. This information helps us understand how visitors use our website.
Google Analytics is a powerful tool that tracks and analyzes website traffic for informed marketing decisions.
Service URL: policies.google.com (opens in a new window)
You can find more information in our Cookie Policy and Privacy Policy.