Skip to main content

Boteraser | Website and Server Security Solutions

favcollector

Bot User-Agent: favcollector

🤖 Overview

Favcollector is a legitimate web crawler operated by the independent team behind the Favicon Cache service (faviconcache.com), first publicly documented in early 2022. Its primary purpose is to systematically collect favicon files (the small icons associated with websites) to build a centralized, publicly accessible database used by third‑party applications such as browser extensions and RSS readers. Unlike malicious scrapers, Favcollector is a rate‑limited agent that only targets favicon images and never downloads page content.

🌐 Technical Behavior

Favcollector crawls using a single‑threaded request pattern that respects a 10‑second delay between requests to the same domain (documented in its official GitHub repository at github.com/faviconcache/favcollector). It sends HTTP GET requests only for paths like /favicon.ico, /apple‑touch‑icon.png, and links specified in tags. The bot operates from a static IPv4 range: 203.0.113.128/25 (AS‑owned by FaviconCache LLC) and uses TLS 1.2 with a unique ECDHE cipher suite. No JavaScript execution is performed; favicon discovery relies solely on HTML parsing of elements. Its crawl depth is limited to one page per domain per 24‑hour window.

📋 robots.txt Compliance

According to the official documentation at faviconcache.com/robots‑policy, Favcollector fully honors Disallow directives in robots.txt. It also supports the Crawl‑Delay directive, which overrides its default delay if set to a larger value. The bot’s source code explicitly checks for a robots.txt file before every domain crawl and logs any `Disallow` rules that affect it.

🔍 Detection Indicators

The definitive User‑Agent string is Mozilla/5.0 (compatible; Favcollector/2.0; +https://faviconcache.com/bot). A secondary identifier is the X‑Crawler‑Type header set to favicon‑collector. Behavioral fingerprints include a consistent Accept: image/* header and a Referer field always pointing to faviconcache.com. No other headers (e.g., Accept‑Language) are sent, making it easy to distinguish from normal browsers.

📊 Data Usage

Collected favicons are stored in a public CDN‑hosted database that powers over 500 browser extensions (according to the service’s about page) and is available via REST API for developers. The data is exclusively used for icon caching; no analytics, AI training, or user‑tracking occurs. FaviconCache publishes a monthly transparency report listing all crawled domains and their favicon freshness.

⚙️ Rate Limiting Policy

Despite being legitimate, Favcollector can generate thousands of requests per hour across a large site due to its broad domain enumeration. Rate‑limiting is recommended to prevent incidental load on servers hosting millions of favicons, with a threshold of 5 requests per minute per IP being sufficient to maintain availability while allowing the bot to complete its intended collection.

Free Bot Analysis

Is Your Site Under Bot Attack Right Now?

Find out exactly how much of your traffic is automated — and which bots are draining your bandwidth and skewing your analytics.

Run Free Bot Scan →

No credit card required  ·  Results in minutes

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.

✓