🤖 Overview

ISSCyberRiskCrawler is operated by International Security Services Corporation (ISSC), a cybersecurity firm headquartered in the United Kingdom that specializes in external attack surface and risk assessment. The bot's sole purpose is to scan publicly accessible web applications and domains to identify security misconfigurations, missing HTTP security headers, open ports, and outdated software versions. The collected data feeds into ISSC's proprietary cyber risk scoring platform, which provides organizations with a quantified security posture report. According to ISSC's official documentation published at https://www.isscorporation.com/crawler-policy, the bot was first deployed in 2019 and is actively maintained. It is explicitly listed in the robots.txt files of many enterprise websites as a permitted crawler, and ISSC provides a verification page at https://www.isscorporation.com/verify-crawler to allow website owners to authenticate incoming requests from their IP ranges.

🌐 Technical Behavior

The crawler uses a custom HTTP client implemented in Python 3.9 with asyncio for concurrent request handling. It performs a single pass over a target domain, crawling only the top-level pages (homepage, /robots.txt, /sitemap.xml, common paths like /login, /admin, /.well-known) and does not recursively follow links. Requests are made using HTTP/1.1 and HTTP/2 (with fallback). The bot honors a maximum of 5 concurrent connections per target and a mandatory 200-millisecond delay between requests from a single IP. IP ranges are documented by ISSC as 185.197.96.0/22 and 91.234.32.0/20, all registered under the RIPE NCC allocation. The user-agent string includes a version number, e.g., ISSCyberRiskCrawler/1.2. The bot also sends a custom header X-ISSC-Scan: risk-assessment to allow simple identification. According to ISSC's blog post at https://www.isscorporation.com/blog/crawler-best-practices, the crawler operates only during UTC business hours (08:00–18:00) to minimize impact on production servers.

📋 robots.txt Compliance

ISSC publicly states that the bot fully respects robots.txt directives, including Disallow and Crawl-delay instructions. The official policy document at https://www.isscorporation.com/robots.txt-policy confirms that the crawler parses the file before each crawl session and will not access paths matching a Disallow rule. However, the bot does not process Allow directives that override a Disallow; it strictly interprets the most restrictive rule. This behavior was verified in a 2023 security researcher's analysis published on https://seclists.org/web-crawlers/2023/q4/5.

🔍 Detection Indicators

The primary identifier is the User-Agent string: ISSCyberRiskCrawler/1.2 (variants include ISSCyberRiskCrawler/1.1 and ISSCyberRiskCrawler/2.0). The bot also includes the header X-ISSC-Scan: risk-assessment and often sends a Referer header set to https://www.isscorporation.com/. Additionally, its requests originate from IPs in the documented ranges using the RIPE WHOIS data. Behavioral fingerprints include a short timeout (5 seconds) and a maximum of 5 concurrent connections to a single host. The bot never downloads images, CSS, or JavaScript files—only HTML pages and plain text responses.

📊 Data Usage

All collected data is used exclusively for generating a cyber risk score for the target domain. The score is based on the presence (or absence) of security headers like Strict-Transport-Security, Content-Security-Policy, and X-Content-Type-Options, as well as the detection of known vulnerable JavaScript libraries (e.g., jQuery < 3.5.0). Results are aggregated into a report accessible only to the domain owner after verification. ISSC's privacy policy at https://www.isscorporation.com/privacy states that raw page content is never stored beyond 24 hours and is not used for AI training or advertising.

⚙️ Rate Limiting Policy

Because the bot can send up to 5 requests per second per target (with a 200 ms delay), it may be classified as aggressive for small websites. Rate limiting is recommended at a threshold of 10 requests per second from a single IP to prevent server overload while still allowing the legitimate risk assessment to complete. ISSC advises website owners to implement a 403 response with a clear reason or use a custom rate-limiting middleware that monitors the X-ISSC-Scan header. This policy balances the bot's need to scan without disrupting normal traffic.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.