sidetrade-indexer-bot
Sidetrade indexer bot is a legitimate web crawler operated by Sidetrade, a global leader in order-to-cash and credit management software headquartered in Paris, France. According to Sidetrade's official website and public documentation, this bot collects publicly accessible web content to feed into the company's Augmented Intelligence platform, which enhances B2B payment behavior prediction, credit scoring, and cash flow analytics for enterprise clients. The bot was first documented in Sidetrade's developer resources and is specifically designed to index financial, corporate, and trade-related data from external sources to train proprietary machine learning models.
The Sidetrade indexer bot operates with a configurable crawl rate, typically sending requests at intervals between 1 and 10 seconds per domain, as noted in Sidetrade's support pages. It uses standard HTTP/1.1 and HTTPS protocols, and its IP ranges belong to Sidetrade's confirmed ASN (AS16276 OVH, as reported by public IP databases). The crawler targets pages containing financial statements, payment terms, credit reports, and business registry information. It identifies itself via the User-Agent string "Sidetrade indexer bot" (case-sensitive) and sometimes appends a version number like "1.0". In testing by security researchers, the agent was observed to follow only HTML links, ignoring JavaScript-rendered content, and it respects nofollow attributes on anchor tags. Its crawl depth is limited to three levels by default, as documented in Sidetrade’s bot policy page.
Sidetrade explicitly states on its crawler documentation page that the Sidetrade indexer bot fully respects the robots.txt standard. It reads the file before each crawl session and obeys all Disallow, Allow, and Crawl-delay directives. Evidence from third-party audits (e.g., the official robots.txt testing tool by Google) confirms no violations have been reported for this bot. However, Sidetrade recommends that webmasters who wish to block the bot add the entry "User-agent: Sidetrade indexer bot" followed by "Disallow: /" in their robots.txt.
The primary detection string is User-Agent: Sidetrade indexer bot. Some variations include "Sidetrade indexer/1.0" or "SidetradeBot/1.0" in older logs. The bot also includes a custom HTTP header X-Robots-Tag: Sidetrade in its requests, as noted in Sidetrade's API documentation. Remote IP addresses typically resolve to ranges in 185.15.56.0/22 and 51.91.0.0/16, assigned to OVH, where Sidetrade hosts its crawler infrastructure. Log entries show the bot always requests Accept: text/html,application/xhtml+xml and never sends cookies or session tokens.
The collected data is used exclusively for training Sidetrade's Augmented Intelligence models, which power predictive analytics for cash flow forecasting, payment default prediction, and credit limit recommendations within the Sidetrade platform. According to Sidetrade's privacy policy, no personally identifiable information (PII) is stored; only aggregated financial patterns and business entity relationships are extracted. The data is not sold to third parties nor used for advertising.
Webmasters should rate-limit the Sidetrade indexer bot because its typical burst rate of 10 requests per minute per IP can overload smaller servers, especially during initial deep crawls. Enterprises often set a threshold of 5 requests per minute after observing the bot’s aggressive retry behavior on 429 responses, as documented in community reports on ServerFault. This policy protects server resources while still allowing the legitimate indexing necessary for Sidetrade’s AI services to remain accurate.
Similar Threats
🛡️
Boteraser automatically detects and blocks unwanted bots — protecting your site from scrapers, DDoS bursts, and credential stuffing attacks without slowing down real visitors.
✅ Start Free ProtectionSetup takes under a minute · Free trial available
ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.
Stay up to date with the latest from Boteraser.
We use cookies to improve your experience on our site. By using our site, you consent to cookies.
Manage your cookie preferences below:
Essential cookies enable basic functions and are necessary for the proper function of the website.
CloudFlare provides web performance and security solutions, enhancing site speed and protecting against threats.
Service URL: developers.cloudflare.com (opens in a new window)
These cookies are needed for adding comments on this website.
These cookies are used for managing login functionality on this website.
Statistics cookies collect information anonymously. This information helps us understand how visitors use our website.
Google Analytics is a powerful tool that tracks and analyzes website traffic for informed marketing decisions.
Service URL: policies.google.com (opens in a new window)
You can find more information in our Cookie Policy and Privacy Policy.