radian6
Bot User-Agent:radian6
🤖 Overview
Radian6 (now part of Salesforce Marketing Cloud since the 2011 acquisition) is a social media monitoring and analytics platform. Its automated crawler, often identified with the User-Agent string "radian6" (all lowercase), systematically collects publicly available content from blogs, forums, news sites, social networks, and review platforms to feed into Salesforce’s Social Studio and other listening products. The bot is operated solely for legitimate market research, brand sentiment analysis, and customer engagement insights—never for malicious purposes.
🌐 Technical Behavior
The Radian6 crawler executes high-frequency, parallelized requests across a distributed set of IP addresses that are typically drawn from cloud providers (e.g., AWS, Google Cloud) and third-party data center ranges. Official documentation from Salesforce’s Developer Center indicates the bot respects a default crawl delay of 1 request per second per domain, but the actual rate can be adjusted by bot operators based on indexing urgency. The crawler fetches both HTML pages and structured data feeds (RSS/Atom) using standard HTTP/1.1 and HTTPS. Observed patterns show it first checks robots.txt before any session begins, then follows internal links recursively while respecting nofollow and noindex meta tags. Its IP ranges are not publicly enumerated in a single list, but third-party logs (e.g., from community forums like WebmasterWorld) report consistent use of AS14618 (Amazon) and AS15169 (Google) prefixes. The bot does not execute JavaScript or load external resources like images; it only consumes raw HTML and CSS-to-text content.
📋 robots.txt Compliance
Salesforce’s official documentation on the Radian6 crawler explicitly states that it obeys standard Disallow directives in robots.txt. Verified by multiple webmaster reports (e.g., a 2019 case study from Search Engine Journal), the bot will cease crawling any path included under Disallow: / within 24 hours of a robots.txt update. However, because the crawler may have cached previous versions, initial requests from a different IP can sometimes bypass a newly added rule until the cache expires. No known CVE entries relate to intentional robots.txt violations by this bot.
🔍 Detection Indicators
The primary User-Agent string is "radian6" (without version number), though variants like "Mozilla/5.0 (compatible; radian6/1.0; +http://www.radian6.com/crawler)" appeared in older logs. Behavioral fingerprints include a low variance in request intervals (typical delta of 0.8–1.2 seconds), a preference for Accept: text/html,application/xhtml+xml headers, and absence of Accept-Language or Accept-Encoding headers. The bot does not set cookies or maintain session state. Reverse DNS lookups of its IPs often resolve to generic hostnames like ec2-*.compute-1.amazonaws.com.
📊 Data Usage
Collected content is processed through natural‑language pipelines (sentiment analysis, entity extraction, trend detection) and aggregated into dashboards within Salesforce Social Studio. The data is used exclusively for commercial social listening—helping brands monitor mentions, gauge public opinion, and identify emerging crises. Radian6 does not resell the raw data or use it for generative AI training; it is solely applied to pre‑compiled analytics reports.
⚙️ Rate Limiting Policy
Despite its legitimacy, Radian6’s crawler can place significant load on small websites because it maintains long‑running sessions across many IPs. Rate‑limiting this bot to a maximum of 5 requests per minute per IP is recommended to preserve server resources while still allowing valuable social listening data to flow—a threshold that aligns with the bot’s own default delay of 1 request per second.
Similar Threats
🛡️
Stop Bots. Save Bandwidth. Protect Revenue.
Boteraser automatically detects and blocks unwanted bots — protecting your site from scrapers, DDoS bursts, and credential stuffing attacks without slowing down real visitors.
✅ Start Free ProtectionSetup takes under a minute · Free trial available
ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.
