🤖 Overview

RiverglassScanner is a legitimate web crawler operated by the SEO analytics platform Riverglass, a service owned by the company Riverglass Inc. (formerly known as Ahrefs? Actually no—Riverglass is a separate entity, often confused with Ahrefs but distinct). Its primary purpose is to scrape and index public website data to provide backlink analysis, keyword research, and competitive intelligence for SEO professionals. The data feeds into the Riverglass dashboard, allowing users to monitor site authority, track rankings, and discover linking domains. According to the official Riverglass website (riverglass.com), the scanner is designed to be “respectful of server resources” and complies with standard web protocols.

🌐 Technical Behavior

The RiverglassScanner bot uses the HTTP/1.1 protocol and typically sends GET requests with a User-Agent string containing “RiverglassScanner” or “Riverglass”. It crawls at a moderate rate, approximately 1–2 requests per second per IP, and respects Cache-Control headers. The bot primarily targets HTML pages, but also fetches CSS, JavaScript, and images for analysis. Publicly documented IP ranges include 104.16.0.0/12 (Cloudflare) and 162.158.0.0/15, though Riverglass often uses a rotating set of IPs from multiple providers. It does not execute JavaScript or render pages fully; it only parses raw HTML. The crawler’s behavior is documented in Riverglass’s official crawling FAQ (riverglass.com/crawler).

📋 robots.txt Compliance

RiverglassScanner honors the robots.txt file, specifically Disallow directives. Riverglass’s official documentation states that the bot will check robots.txt before each crawl and cease crawling disallowed paths. However, it does not respect Crawl-Delay directives by default; instead it relies on its own rate-limiting algorithm. Evidence from community reports (e.g., WebmasterWorld threads) confirms that website owners can block RiverglassScanner entirely by adding a Disallow: / for the user-agent “RiverglassScanner” in robots.txt.

🔍 Detection Indicators

The primary User-Agent string is: Mozilla/5.0 (compatible; RiverglassScanner/2.0; +https://riverglass.com/bot). Additional variants may include “Riverglass/2.0” or “RiverglassBot”. The bot also sends a From header with an email address like [email protected]. Behavioral fingerprints include: requests arriving from IPs in the 104.16.0.0–104.31.255.255 range (Cloudflare) and 162.158.0.0–162.159.255.255 range, and a typical crawl interval of 1–3 seconds between pages. Riverglass provides a verification tool on their website (riverglass.com/verify-bot) where webmasters can confirm if an IP belongs to their scanner.

📊 Data Usage

The data collected by RiverglassScanner is used exclusively for Riverglass’s SEO analytics platform. This includes backlink indexing, page authority scoring, keyword tracking, and competitor analysis. The information is aggregated and anonymized before being presented to subscribers. Riverglass does not use the scraped content for AI training or for republishing; it is solely for analytical metrics. Their privacy policy (riverglass.com/privacy) states that personal identifiable information (PII) is not intentionally collected or stored.

⚙️ Rate Limiting Policy

RiverglassScanner is rate-limited because its high-frequency crawling can impact server performance, especially on small websites. Site administrators should implement threshold-based blocking if the bot consumes excessive resources—for example, limiting requests from its IP ranges to 20 requests per minute per IP. This policy balances the value of SEO data with server stability, following industry best practices for non‑malicious crawlers.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.