🤖 Overview

TurnitinBot is an automated web crawler operated by Turnitin, LLC, a company specializing in plagiarism detection and academic integrity solutions. First documented around 2016, its primary purpose is to systematically index publicly accessible web content—including academic papers, blogs, news articles, and other textual resources—for ingestion into Turnitin’s proprietary similarity-checking database. The collected data powers features such as the Originality Report, which cross-references student submissions against a vast repository of existing internet works.

🌐 Technical Behavior

The bot exhibits polite crawling behavior, typically respecting a moderate request rate to avoid overwhelming servers. According to Turnitin’s official crawler information page (https://turnitin.com/robot/crawlerinfo.html), it defaults to one request every 10 seconds but may adjust dynamically based on server responses and available bandwidth. It connects via standard HTTP/1.1 on ports 80 and 443, using the User-Agent string TurnitinBot/1.5 (common variants include TurnitinBot/1.6 and older TurnitinBot/0.1). IP ranges are not publicly fixed; the crawler uses a pool of addresses owned by Turnitin’s cloud infrastructure providers, primarily Amazon Web Services (AWS) and, in some cases, Turnitin’s own Class C subnets. It does not support parallel fetching; each crawl request waits for a complete response before proceeding.

📋 robots.txt Compliance

Turnitin explicitly states that its bot fully honors robots.txt directives. The official documentation confirms that Disallow rules are parsed and obeyed, and the crawler will also respect Crawl-Delay instructions if present. There is no known evidence of TurnitinBot ignoring robots.txt, and the company encourages site administrators to use the file to restrict access to sensitive areas.

🔍 Detection Indicators

The primary identification is the User-Agent string TurnitinBot/1.5 (or a version variant), sometimes accompanied by a comment linking to https://turnitin.com/robot/crawlerinfo.html. Another header often included is From: [email protected]. Behavioral fingerprint: the bot sends a single HTTP GET request per URL, does not attempt to parse JavaScript-rendered content, and its referrer field is typically blank or set to the previously crawled page. It may also present a custom X-Robots-Tag header to signal compliance with meta directives.

📊 Data Usage

Collected web pages are stored in Turnitin’s centralized content repository and are used exclusively for plagiarism detection. When a user submits a document via Turnitin’s writing tools or integration platforms, the submission is compared against the crawled database to identify text overlap. The data is not repurposed for advertising, AI training, or third-party analytics; its sole function is academic integrity verification.

⚙️ Rate Limiting Policy

Although TurnitinBot is designed to be non‑aggressive, it is still subject to rate‑limiting because its cumulative crawl volume—across thousands of target domains—can approach thresholds indicative of automated scraping. Administrators may implement threshold‑based blocking (e.g., >20 requests per minute from the same IP) to prevent unintended resource exhaustion while still permitting legitimate indexing.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.