🤖 Overview

its-learning crawler is a legitimate web crawler operated by itslearning AS, a Norwegian educational technology company headquartered in Bergen that develops the itslearning Learning Management System (LMS). The crawler indexes external web resources—such as pages, PDFs, and images—linked by educators within course content, enabling full-text search inside the platform. Its official documentation confirms the bot is institution-configured and used solely for educational indexing.

🌐 Technical Behavior

The bot issues standard HTTP/1.1 GET requests, respects Cache-Control and ETag headers, and does not execute JavaScript. Crawl frequency is moderate and configurable per institution, typically ranging from hourly to daily checks. IP ranges belong to itslearning’s hosting infrastructure, primarily AWS EC2 instances in EU regions, grouped under autonomous system AS20473 (Host Europe) or directly registered to itslearning AS. The bot identifies itself with the User-Agent its-learning-crawler/1.0 and a From header of [email protected].

📋 robots.txt Compliance

Based on itslearning’s support pages and public bug tracker, the crawler fully honors robots.txt directives, reading Disallow rules before every requested path. Institutions can also set granular exclusion rules via the LMS admin panel. No CVE or security advisory has reported non-compliance; the bot is considered well-behaved per standard web robot etiquette.

🔍 Detection Indicators

Primary fingerprint: User-Agent its-learning-crawler/1.0 with possible variations like itslearning-crawler/1.0. It sends a Referer header of https://itslearning.com/ and requests only text/html, image/*, and application/pdf MIME types. No cookies or JavaScript are used. Network logs show requests from AWS EC2 IP ranges with a consistent pattern of low frequency (2–5 requests per minute) and a distinct User-Agent containing “learning”.

📊 Data Usage

Collected data is used exclusively within the itslearning LMS for indexing and search purposes. Fetched content generates thumbnails, extracts text for full-text search, and validates link health. No data is sold or used for external AI training; itslearning’s privacy policy states cached resources are deleted when the linked content is removed or the course expires.

⚙️ Rate Limiting Policy

The crawler is rate-limited because its aggregate requests from thousands of courses can spike demand on small servers. Standard recommendation is to allow up to 10 requests per minute per IP and throttle above that, as the bot normally runs at 2–3 rpm. This threshold protects web servers while enabling essential LMS indexing.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.