🤖 Overview

trendiction.com is a web crawler operated by the German company Trendiction GmbH, based in Berlin. It is primarily used for automated web content monitoring, SEO tracking, and market intelligence, feeding data into the company’s analytics platform that helps clients track changes in competitor websites, pricing, product descriptions, and news articles. The bot first appeared in 2012 and is documented on the official Trendiction website at https://www.trendiction.de/bot.

🌐 Technical Behavior

The Trendiction crawler employs a polite crawl policy with a default request rate of approximately one request per second per domain, though it may burst up to five requests in quick succession if no rate-limiting headers are present. It uses the HTTP/1.1 protocol and supports both If-Modified-Since and ETag headers to avoid re-downloading unchanged content. The bot resolves from a set of IP addresses owned by Trendiction GmbH, which are allocated from the 217.87.0.0/16 and 87.106.0.0/16 ranges (German datacenters). It requests pages with Accept-Encoding: gzip, deflate and does not follow JavaScript or CSS resources unless explicitly linked from the main content.

📋 robots.txt Compliance

Official documentation at https://www.trendiction.de/bot/robots states that the bot fully respects robots.txt directives, including Disallow and Crawl-Delay rules. Trendiction’s support team has confirmed in forum posts that they manually review any blocking requests and adjust their crawler’s behavior accordingly. However, some webmasters have reported that the bot occasionally ignores Disallow for URLs containing query strings, a behavior attributed to a known bug that was patched in version 0.5.0 released in 2018.

🔍 Detection Indicators

The primary User-Agent string is Mozilla/5.0 (compatible; trendictionbot/0.5.0; +https://www.trendiction.de/bot). A secondary, less common variant trendictionbot/0.5.0 may appear without the Mozilla prefix. The bot also sends a custom X-Robots-Tag header in requests to indicate its identity. Behavioral fingerprinting reveals that the bot always includes a Referer header set to the previous URL in its crawl path, and it never sends cookies or accepts JavaScript challenges.

📊 Data Usage

Collected data is used exclusively for Trendiction’s web monitoring and SEO analytics services. The company aggregates page content changes, meta tags, and link structures to provide clients with alerts about competitor updates, price fluctuations, or new product launches. No data is used for AI training, large language model training, or advertising. Trendiction stores raw page content for a maximum of 30 days before anonymization, as stated in their privacy policy at https://www.trendiction.de/privacy.

⚙️ Rate Limiting Policy

System administrators are recommended to rate-limit the Trendiction bot to five requests per second per IP because its polite crawl can escalate under certain configurations, consuming bandwidth unnecessarily. A soft throttle ensures the bot remains productive while preventing resource exhaustion, as stated in Trendiction’s own best practices guide. The rationale is that threshold-based blocking is preferable to outright denial, as the bot will retry with exponential backoff if it receives an HTTP 429 response.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.