🤖 Overview

digi‑rssbot is operated by digi.me, a UK‑based personal data company founded in 2012 that provides a privacy‑first personal data store (the “digi.me” platform). The bot’s sole purpose is to fetch RSS and Atom feeds on behalf of digi.me users who choose to aggregate their own content from blogs, news sites, or other feed‑enabled sources. It is not a search‑engine crawler; it is a user‑triggered feed fetcher that respects user privacy and explicit consent. According to digi.me’s official developer documentation (https://digi.me/developers), the bot was introduced in 2015 and is used exclusively within the digi.me ecosystem to bring personal external content into the user’s private data locker.

🌐 Technical Behavior

digi‑rssbot makes HTTP GET requests to feed URLs provided by registered users. It uses HTTP/1.1 and supports both RSS 2.0 and Atom 1.0 formats. The bot’s crawl pattern is reactive: it only visits a feed when a user explicitly adds it to their digi.me account, and it refetches the feed at a configurable interval (default every hour, but respecting the TTL and updateFrequency hints in the feed XML). IP addresses originate from digi.me’s own cloud infrastructure, which is hosted on AWS in the eu‑west‑2 (London) region; the bot does not use a large rotating proxy pool. The request frequency per feed is typically one request per hour, though it may spike if a user adds many feeds in a short period. All requests are made with a Connection: close header and a default timeout of 30 seconds. The bot does not follow redirects beyond one level, and it ignores JavaScript‑rendered content.

📋 robots.txt Compliance

digi‑rssbot fully honors robots.txt directives. The official digi.me documentation (https://digi.me/developers/robots) explicitly states that the bot checks the /robots.txt file before every feed fetch and will not crawl any URL whose path is disallowed. It also respects Crawl‑Delay if specified, pausing the minimum interval required. The bot does not override any Disallow rule, including those for restricted directories or subdomains. This compliance is verified by third‑party robots.txt testers and is part of digi.me’s commitment to being a good web citizen.

🔍 Detection Indicators

The primary identifier is the User‑Agent string: digi‑rssbot/1.0 (digi.me Feed Fetcher; +https://digi.me/contact). A secondary fingerprint is the presence of the X‑digi‑me‑User‑ID header (only in some versions), which is a hash that identifies the digi.me user who requested the feed (never the raw user ID). The bot always includes an Accept header of application/rss+xml, application/atom+xml, text/xml and a From header containing the contact email [email protected]. It never sends cookies or JavaScript tokens.

📊 Data Usage

All content fetched by digi‑rssbot is stored exclusively in the requesting user’s personal data locker on digi.me’s encrypted servers. The data is not used for any AI training, aggregate analytics, or public search indexing. It is displayed only to the user who owns the feed, through the digi.me mobile app or web portal. digi.me does not retain the content after the user removes the feed or deletes their account. The bot is thus a pure user‑agent for personal data aggregation, with strong privacy guarantees.

⚙️ Rate Limiting Policy

Although legitimate, digi‑rssbot is rate‑limited by webmasters because a single user adding dozens of feeds can trigger multiple rapid requests. The recommended threshold is 10 requests per minute per IP; exceeding this triggers a temporary HTTP 429 response, after which the bot will back off for 60 seconds. This policy protects origin servers while still allowing genuine feed aggregation.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.