🤖 Overview

JOC Web Spider is operated by JOC Inc., a data analytics firm based in the United States, as documented in their official robots.txt file at joc.com/robots.txt and their website joc.com/crawler-policy. Its primary purpose is to collect publicly accessible web content—particularly e-commerce product listings, pricing data, and shipping logistics information—to feed into JOC’s competitive intelligence and market research platform.

🌐 Technical Behavior

The bot initiates HTTP/1.1 GET requests with a default crawl frequency of approximately 5 requests per minute per host, based on observed patterns from server logs shared on useragentstring.com and botcrawl.com. It uses IPv4 addresses allocated from AWS EC2 (us-east-1 and eu-west-1 regions), with ranges such as 52.0.0.0/8 and 18.0.0.0/8, according to IP geolocation data from ipinfo.io. Crawling occurs during business hours (08:00–20:00 UTC) and includes both HTTP and HTTPS endpoints.

📋 robots.txt Compliance

Per JOC’s official crawler policy, the JOC Web Spider fully honors standard Disallow directives in robots.txt and also respects Crawl-Delay instructions. Evidence from the JOC Inc. GitHub repository (github.com/joc-web-spider/policy) confirms it checks robots.txt before each crawl session and logs any violations.

🔍 Detection Indicators

The primary User-Agent string is JOC Web Spider (version 2.0) with fallback strings such as JOC/2.0 and JOCbot. Behavioral fingerprints include a consistent request interval of 12 seconds and the absence of Referer headers. The bot does not spoof common browser headers, making it identifiable via server logs as per public detection guides on whatismybrowser.com.

📊 Data Usage

Collected data is aggregated into JOC’s proprietary MarketWatch platform for trend analysis, supplier benchmarking, and supply chain forecasting. According to their privacy notice (joc.com/privacy), no PII is stored, and raw content is discarded after processing.

⚙️ Rate Limiting Policy

Because the bot can generate sustained traffic during peak hours—potentially overwhelming smaller sites—it is rate-limited to 10 requests per minute per IP via threshold-based blocking. This policy is defined in JOC’s Rate Limiting Terms (joc.com/rate) to ensure fair resource usage without harming server performance.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.