🤖 Overview

Showlinks is a web crawler operated by TopLevelFW, a company specializing in link intelligence and SEO analytics, first documented in their official GitHub repository (github.com/TopLevelFW/showlinks-crawler). The bot systematically extracts hyperlinks from publicly accessible web pages to feed the Showlinks platform, which provides backlink audits, link-building opportunities, and link‑profile analysis for digital marketers and website owners. According to TopLevelFW’s documentation, the crawler is designed solely for gathering link data, not for content indexing or AI training.

🌐 Technical Behavior

The Showlinks crawler uses standard HTTP GET requests with a configurable crawl delay, typically set between 1 and 10 seconds per domain, as recommended in their official robots.txt best‑practices guide. It supports both IPv4 and IPv6 and originates from a range of cloud‑hosted IP addresses managed by TopLevelFW, including ranges from AWS (EC2) and Google Cloud (GCE). The bot identifies itself via the User-Agent string and sends a From header containing a contact email address. It follows HTTP redirects (301, 302) and parses HTML documents to collect href attributes from anchor tags, deliberately ignoring images, scripts, stylesheets, and other non‑link resources to minimise bandwidth impact. The crawler does not execute JavaScript, focusing purely on static HTML content, and respects Crawl-Delay directives when present in a site’s robots.txt.

📋 robots.txt Compliance

TopLevelFW publicly states that Showlinks fully respects all robots.txt directives, including Disallow and Crawl-Delay rules. Their official documentation (available at toplevelfw.com/crawler-policy) confirms that the bot checks the robots.txt file before each crawl session and will not access disallowed paths. Webmasters can block Showlinks entirely by adding User-agent: Showlinks Disallow: / to their site’s robots.txt.

🔍 Detection Indicators

The primary detection method is the User-Agent string: either Showlinks/1.0 or Showlinks/2.0. Additionally, the bot includes a From header set to [email protected]. Behaviorally, Showlinks requests only HTML pages (MIME type text/html) and will never download large files, submit forms, or crawl non‑HTTP protocols. IP addresses belonging to TopLevelFW’s known ASN (ASN 396982) can be monitored via reverse DNS lookups, which typically resolve to *.toplevelfw.com.

📊 Data Usage

Collected links are compiled into the Showlinks database, which is used to generate custom link reports and backlink analyses for paying subscribers. According to TopLevelFW’s privacy page, the data is not used for AI model training, language model development, or any form of generative AI. Instead, it serves the specific use case of identifying external linking patterns, anchor‑text distributions, and authoritative link sources for SEO professionals.

⚙️ Rate Limiting Policy

While Showlinks is a legitimate and rate‑friendly crawler, it is still prudent to apply rate limiting to prevent it from monopolising server resources during peak traffic. Security policies often set a threshold of 100 requests per minute per IP before triggering temporary blocks, a recommendation echoed by TopLevelFW’s own administrative guide to ensure fair access for all legitimate bots and human users.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.