🤖 Overview

ExtLinksBot is a legitimate web crawler operated by LinkResearchTools (LRT), an Austrian SEO software company founded in 2011. Its purpose is to scan publicly accessible web pages to build an index of external outbound links, used for backlink analysis, broken link detection, and competitor profiling within the LRT platform. LRT’s flagship product, the LinkResearchTool, relies on this crawler to deliver actionable link intelligence. The bot has been active since at least 2013 and is widely recognized in the SEO community.

🌐 Technical Behavior

The crawler sends HTTP/1.1 and HTTPS requests at a moderate rate of 10 to 50 pages per minute, adjusting based on server response times. It follows standard hyperlinks, skips links with rel="nofollow", and does not access password-protected or non-public directories. IP addresses originate from a static pool allocated to LRT’s Austrian infrastructure, though exact ranges are not publicly listed. The bot requests only text/html content, does not execute JavaScript, and respects HTTP status codes. It also honors the Link header for pagination, follows redirects (301, 302), and respects meta refresh tags. The bot sends standard Accept and Accept-Language headers. Crawling uses multiple concurrent threads but throttles itself when encountering slow responses.

📋 robots.txt Compliance

According to LRT’s official documentation, ExtLinksBot fully respects robots.txt. Site owners can block it with a Disallow: / directive for ExtLinksBot. The bot also supports Crawl-Delay directives, though no default delay is advertised. LRT publishes its robots.txt policy on their website at linkresearchtools.com/robots.txt.

🔍 Detection Indicators

The User-Agent string is ExtLinksBot/1.0 (with version numbers like 1.1 or 2.0 observed). Each request includes a From email header: [email protected]. No other custom headers are present. The bot’s IP addresses resolve to hostnames under lrtnetwork.com. Behavioral fingerprinting reveals consistent requests for HTML pages only, with no image, CSS, or script downloads. The bot sometimes requests gzip-encoded content.

📊 Data Usage

Collected link data is used exclusively within the LinkResearchTools suite to generate link profile reports, detect broken backlinks, and analyze competitor external linking strategies. LRT states that page content is not stored beyond the indexing needed for link extraction, and data is not used for AI training, search indexing, or sold to third parties. No personally identifiable information is intentionally harvested. Subscribers of LRT can query the index for specific domains or patterns, and data is refreshed periodically to maintain accuracy.

⚙️ Rate Limiting Policy

While not malicious, ExtLinksBot’s crawling can be aggressive on smaller websites. Rate limiting with a threshold of 100 requests per minute per IP is recommended to protect server resources while still permitting legitimate link data collection for SEO analysis. System administrators are advised to monitor logs for sudden spikes and apply rate limiting if the bot causes performance issues.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.