🤖 Overview

taxinomiabot is a legitimate web crawler operated by Taxinomia, a French company specializing in product data aggregation and e‑commerce intelligence. Its primary purpose is to collect publicly available product information from online retailers for use in price comparison engines, market analysis, and catalog enrichment services. The bot feeds data into Taxinomia’s platform, which provides structured product data to merchants and consumers, as documented on the official Taxinomia website (https://www.taxinomia.com).

🌐 Technical Behavior

The crawler performs periodic scanning of e‑commerce sites, focusing on product pages, category listings, and pricing data. It uses HTTP/1.1 or HTTP/2 protocols and identifies itself with the User‑Agent string taxinomiabot/1.0. Taxinomia does not publish a fixed IP range, but the bot originates from a set of IPs geolocated in France. Requests are sent sequentially with a default delay of at least 1 second between pages, as stated in the official bot documentation at https://www.taxinomia.com/bot.html. The crawler supports conditional requests via If‑Modified‑Since headers to reduce server load, and it avoids crawling during peak hours based on a configurable schedule. It does not use distributed IPs or concurrent connections from multiple hosts, making it easier to identify in access logs.

📋 robots.txt Compliance

Taxinomia claims that taxinomiabot fully honors robots.txt directives, including Disallow rules. However, some webmaster reports (e.g., on community forums) note that the bot occasionally ignores the Crawl‑Delay directive, relying instead on its internal rate‑limiting logic. The company recommends site owners explicitly disallow sensitive paths to prevent access. Official documentation confirms that the bot reads robots.txt at the start of each crawl session and respects standard restrictions.

🔍 Detection Indicators

The primary identifier is the User‑Agent string taxinomiabot/1.0, sometimes with a comment like +(https://www.taxinomia.com/bot.html). The bot also sends a From header with the email [email protected] for contact. Behavioral fingerprints include sequential page requests within a single domain, a consistent request interval of 1‑2 seconds, and no referrer header. Log entries show a pattern of fetching product schema markup (JSON‑LD or microdata) and avoiding static assets like images or CSS.

📊 Data Usage

All collected data is used solely for Taxinomia’s product indexing and market intelligence platform. This includes product names, prices, availability, images, and descriptions, which are aggregated to power price comparison sites, affiliate networks, and analytics dashboards for retailers. Taxinomia explicitly states that it does not use the data for AI training or generative models, and it respects copyright by only indexing publicly accessible content.

⚙️ Rate Limiting Policy

Despite its legitimacy, taxinomiabot is often rate‑limited on high‑traffic e‑commerce sites to prevent server overload. The standard policy is a threshold of 100 requests per minute, after which the bot is throttled or temporarily blocked. This balances the benefit of product data collection with the need to maintain site performance for human users.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.