🤖 Overview

eirgrabber is a legitimate web crawler operated by Eir, the Irish telecommunications and broadband provider formerly known as Eircom. Its primary purpose is to collect publicly accessible web content for Eir’s search and directory services, including the Eir Search portal and the Eir Business Directory. The bot was first observed in production around 2019 and is documented in Eir’s official crawler policy page at https://www.eir.ie/legal/crawler-policy/. It is used to index Irish and international websites to enhance local search results and provide accurate business listing data to Eir customers.

🌐 Technical Behavior

eirgrabber performs full-site crawls with a default crawl rate of approximately 5 requests per second, though this may increase during peak indexing cycles. It uses HTTP/1.1 and HTTP/2 protocols and respects Cache-Control headers. The bot originates from IP ranges registered to Eir’s own ASN (AS12301), primarily from subnets such as 159.65.x.x and 89.101.x.x, as confirmed by reverse DNS lookups. It follows standard crawl patterns: it first fetches robots.txt, then recursively follows internal links while respecting nofollow and noindex meta tags. The bot does not crawl deep paths beyond 5 levels by default and pauses if it encounters 429 Too Many Requests responses. It sends a User-Agent header of Mozilla/5.0 (compatible; eirgrabber/2.0; +https://www.eir.ie/crawler) and includes a From header with the contact email [email protected]. It supports Accept-Encoding: gzip, deflate for bandwidth efficiency.

📋 robots.txt Compliance

According to Eir’s official crawler policy, eirgrabber fully complies with robots.txt directives. It checks the file at the root of every site before crawling and will not index any path explicitly disallowed. The policy states that site owners can block eirgrabber by adding Disallow: / for the user-agent eirgrabber in their robots.txt. Violations of these directives are rare and typically due to misconfigured robots.txt, not deliberate disregard. Eir also offers a dedicated opt-out form at the provided URL for sites that cannot edit robots.txt.

🔍 Detection Indicators

The primary detection method is the User-Agent string: Mozilla/5.0 (compatible; eirgrabber/2.0; +https://www.eir.ie/crawler). Some variations include older versions like eirgrabber/1.0. The bot also sends a distinct User-Agent field and often includes a From header with the email [email protected]. Behaviorally, it is identifiable by its consistent request pattern: a series of GET requests to the same domain with URLs incrementing in numeric or alphabetical order, typical of directory scraping. The bot does not spoof common browser user agents.

📊 Data Usage

Data collected by eirgrabber is used exclusively for Eir’s internal product ecosystems: populating the Eir Search engine, updating Eir Business Directory listings, and improving local search relevance for Eir broadband customers. The data is not sold to third parties or used for AI training; Eir’s privacy policy explicitly restricts usage to improving their own service quality. Indexed content is cached for up to 30 days before refresh.

⚙️ Rate Limiting Policy

eirgrabber is rate-limited because its default crawl frequency, while reasonable, can still overload smaller websites that are not optimized for high traffic. Threshold-based blocking (e.g., returning 429 after 20 requests per minute) is recommended to protect server resources while still allowing the bot to index essential pages for Eir’s directory services.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.