🤖 Overview

dotspotsbot is the automated crawler operated by DotSpots (dotspothq.com), a website change‑detection and monitoring service launched in 2021. Its sole purpose is to periodically fetch web pages on behalf of paying customers so the platform can compare snapshots and notify users when content changes — for example, price drops, policy updates, or new job postings.

🌐 Technical Behavior

The bot performs HTTP GET requests using HTTP/1.1 with standard headers (Accept, Accept-Encoding). By default it checks pages every 15‑60 minutes, though the interval can be configured by the user. IPs are drawn from Amazon Web Services (specifically the us-east-1 and eu-west-1 regions) and from DigitalOcean datacenters; the official documentation at https://docs.dotspothq.com/dotspotsbot lists a CIDR range of 54.162.0.0/16 and several /24 blocks. The crawler does not execute JavaScript or render WebGL, so pages depending on dynamic client‑side rendering may not trigger change alerts. Requests include a unique X-DotSpots-Request-ID header per session to help site operators correlate traffic.

📋 robots.txt Compliance

According to DotSpots’ public robots.txt policy (published at https://dotspothq.com/robots.txt), dotspotsbot fully honors Disallow directives. The official documentation states that the bot checks for a /robots.txt file at the start of each crawl cycle and caches the rules for up to one hour. No evidence of intentional rule violations has been reported in security advisories or webmaster forums as of 2025.

🔍 Detection Indicators

The primary User‑Agent string is dotspotsbot/1.0 (case‑sensitive) and appears without additional platform or version suffixes. A secondary form DotSpotsBot/1.0 has been observed in older logs. The bot also sends a distinct User-Agent header containing the substring dotspots and sets a From header with the email address [email protected]. Behavioral fingerprints include a strict 30‑second timeout between requests to the same domain and a maximum of 2 concurrent connections per host.

📊 Data Usage

Collected HTTP responses are stored temporarily in Amazon S3 and compared using a diffing algorithm. The results are displayed to the customer who initiated the monitor; raw page content is never used for AI training, advertising, or resale. DotSpots’ privacy policy (available at https://dotspothq.com/privacy) states that historical snapshots are retained for 90 days and then deleted.

⚙️ Rate Limiting Policy

Because dotspotsbot can generate a high volume of requests when customers monitor many pages at short intervals, it is legitimately rate‑limited by many web applications to protect origin server resources. A threshold of 50 requests per hour per IP is a common policy that distinguishes benign monitoring from accidental overload, while still allowing the bot to complete its change‑detection function for subscribed users.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.