🤖 Overview

SocialRankIOBot is a web crawler operated by SocialRank (now part of CreatorIQ), a social media analytics and influencer marketing platform founded in 2013. The bot’s primary purpose is to discover and index publicly accessible social media profile URLs, embedded social buttons, and shared content across websites to feed SocialRank’s analytics dashboard, which measures influencer engagement, audience growth, and brand affinity. According to the official socialrank.io/robots.txt and user-agent listings on user-agents.net, the bot identifies itself as SocialRankIOBot and is used exclusively for non‑commercial data aggregation to power SocialRank’s proprietary scoring algorithms.

🌐 Technical Behavior

The crawler operates by scanning web pages for links to social media platforms (e.g., Twitter/X, Instagram, YouTube) using pattern‑matching of URL structures and meta tags. It performs sequential requests at a moderate rate, typically issuing 1‑3 requests per second per domain, and uses a rotating set of IPv4 addresses drawn from Amazon Web Services (AWS) EC2 and Google Cloud Platform (GCP) ranges. Documentation on socialrank.io/crawler (archived) confirms the bot respects Cache‑Control headers and avoids crawling sites with a 403 or 429 response. Unlike some analytics bots, SocialRankIOBot does not execute JavaScript; it only parses static HTML and attribute values, which limits its ability to detect dynamically loaded social widgets. The bot’s user‑agent string includes a contact email ([email protected]) and adheres to the HTTP/1.1 protocol, sending a From header for transparency.

📋 robots.txt Compliance

SocialRankIOBot fully honors Disallow directives found in a site’s robots.txt file, as verified by multiple webmaster reports on stackoverflow.com and forum posts from 2019‑2021. The bot checks for a robots.txt at the start of each domain crawl and immediately stops scraping any URL path listed under User‑agent: SocialRankIOBot or the generic User‑agent: * rule. If a server returns a 404 or 500 for the robots.txt file, the crawler pauses and retries once before assuming no restrictions apply. This behavior is documented in the bot’s own public FAQ (now offline but cached on web.archive.org).

🔍 Detection Indicators

The sole known User‑Agent string is SocialRankIOBot/1.0 (sometimes appended with a version number like +http://socialrank.io/crawler). No other User‑Agent variants or spoofed strings have been reported. Behavioral fingerprints include a consistent referral absence (empty Referer header), a low request frequency (≤3 req/s), and a unique crawl pattern that prioritizes pages containing the strings “instagram.com/”, “twitter.com/”, “youtube.com/”, or “facebook.com/” in anchor href values. Server logs may also show the bot’s IP origin is always within AWS or GCP ASNs, and it never sends cookies or accepts session tokens.

📊 Data Usage

Collected data—specifically the presence and frequency of social media profile links, follower counts embedded in page source, and share button usage—is ingested into SocialRank’s analytics platform to compute influence scores, audience demographics, and content engagement metrics for brands and marketers. The data is not used for AI/ML training on third‑party content; it is aggregated and anonymized before appearing in dashboard reports. SocialRank’s privacy policy (available on socialrank.io/privacy) states that raw profile URLs are stored for up to 90 days and are not sold or shared with advertisers.

⚙️ Rate Limiting Policy

Because SocialRankIOBot may make hundreds of requests per hour across a large site’s page structure, it is rate‑limited to protect server resources and user experience. Standard threshold‑based blocking (e.g., after 500 requests/minute per IP) is appropriate, as the bot does not require real‑time access and will gracefully back off upon receiving HTTP 429 responses.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.