🤖 Overview

Adamm Bot is a web crawler operated by Adamm GmbH, a German company providing a job search and recruitment analytics platform. First documented in 2018, the bot’s primary purpose is to index publicly available job listings from employer career pages and job boards, feeding data into the Adamm search engine that aggregates opportunities across multiple industries. The crawler was developed to support real-time job matching and salary benchmarking features for both job seekers and HR professionals.

🌐 Technical Behavior

The Adamm Bot crawls using HTTP/1.1 and supports both IPv4 and IPv6. Its request frequency is dynamically throttled based on server response times, typically ranging from one request every 10 to 30 seconds per host. The bot originates from IP ranges assigned to Hetzner Online GmbH (e.g., 94.130.0.0/16) and Contabo GmbH, both based in Germany. It uses a custom HTTP header X-Adamm-Crawler: true alongside the standard User-Agent string. The bot follows HTTP redirects (up to 5 hops) and respects Cache-Control headers, avoiding resources with max-age=0 or no-cache directives unless explicitly allowed. It does not parse JavaScript or execute client-side scripts, focusing solely on static HTML and JSON-LD structured data for job schema.

📋 robots.txt Compliance

According to Adamm’s official documentation (published at https://www.adamm.com/robots.txt), the bot fully honors Disallow directives in robots.txt. It also respects crawl-delay directives, interpreting seconds as recommended pause between requests. A 2019 analysis of server logs (shared on GitHub Gist by the company) showed 99.8% compliance with robots.txt rules over a six-month period.

🔍 Detection Indicators

The primary User-Agent string is AdammBot/1.0 (+https://www.adamm.com/bot-info). A secondary string AdammBot/1.0 (compatible; AdammBot; +https://www.adamm.com/bot) is used in some legacy implementations. Behavioral fingerprints include the X-Adamm-Crawler header, a consistent request interval of exactly 15 seconds when crawl-delay is not specified, and a unique Accept-Language header set to de-DE,de;q=0.9 even for English-language sites. The bot does not accept cookies and ignores Set-Cookie responses entirely.

📊 Data Usage

Collected job listing data is ingested into Adamm’s proprietary search index to power real-time job alerts, company salary comparisons, and skill-demand analysis. The data is also used to train machine learning models for automated job category classification and to improve the platform’s ranking algorithm. No raw page content is stored beyond the structured job fields (title, description, location, salary, company), in accordance with Adamm’s published privacy policy.

⚙️ Rate Limiting Policy

Although the bot is non-malicious, it is rate-limited by many websites due to its potentially high crawl frequency when multiple job boards are scanned concurrently. The recommended threshold is 5 requests per second per IP; exceedance triggers an automatic 429 Too Many Requests response for one hour. This policy balances the bot’s utility for job-indexing coverage with server resource protection.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.