🤖 Overview

art-online com is a legitimate web crawler operated by art-online.com, a German online art marketplace and publisher. Its primary purpose is to index publicly available art-related content, including artist portfolios, gallery listings, and art news articles, to feed into the art-online.com search engine and recommendation system. According to the company’s official website, the bot is designed to aggregate metadata from third-party art platforms to enrich its own database and provide users with comprehensive art search results.

🌐 Technical Behavior

The crawler uses standard HTTP/1.1 and HTTPS protocols, typically making requests at a moderate rate of one request every 5 to 10 seconds per domain, as documented in the art-online.com developer blog. It respects standard robots.txt Crawl-delay directives and may also send a From header with an administrative email address for site owners to contact. The bot’s IP ranges are registered under the ASN AS51167 (art-online.com GmbH) and span a limited block of static IPv4 addresses, as confirmed by whois lookups from 2024. It primarily crawls URLs matching art-related paths (e.g., /gallery, /artist, /exhibition) and avoids deep pagination beyond 2,000 links per site.

📋 robots.txt Compliance

Official documentation on the art-online.com robots.txt guidance page states that the crawler fully honors Disallow directives and the Crawl-delay value specified in robots.txt. Testing by independent researchers in 2023 confirmed no violations of disallowed paths, although some operators reported that the bot occasionally retried blocked URLs after a 24-hour cooldown period.

🔍 Detection Indicators

The bot identifies itself with the User-Agent string Mozilla/5.0 (compatible; art-online-bot/1.0; +https://art-online.com/bot), and sometimes a secondary string art-online-crawler (art-online.com). It does not modify the User-Agent dynamically, making it straightforward to detect. Behavioral fingerprints include a consistent request pattern with Accept: */* and a low rate of concurrent connections—rarely more than two simultaneous requests to the same host.

📊 Data Usage

Collected data—including artwork titles, artist names, exhibition dates, and image metadata—is used to populate the art-online.com search index, which includes over 2 million art objects as of 2025. The company publicly states that the content is not used for AI training or resale, only for improving search relevance and recommendation algorithms within its own platform.

⚙️ Rate Limiting Policy

Because the bot is a legitimate aggregator that respects crawl delays, rate limiting is not strictly necessary, but administrators may apply a threshold of 5 requests per second per IP to prevent accidental overload. This policy ensures fair resource allocation while still allowing the bot to index large art databases.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.