🤖 Overview

Atlassian-bot is a legitimate web crawler operated by Atlassian Corporation Plc, the Australian enterprise software company behind products like Jira, Confluence, Bitbucket, and Trello. Its primary purpose is to crawl publicly accessible web content—such as developer documentation, forums, and marketplace listings—to feed into Atlassian’s internal search indexing, product intelligence features, and AI-powered assistance tools like Atlassian Intelligence. The bot was introduced around 2021 and is described in Atlassian’s official crawler documentation as a “friendly crawler” that respects site policies and is used exclusively for improving user experience within Atlassian applications.

🌐 Technical Behavior

Atlassian-bot employs standard HTTP/1.1 requests with a GET method and respects robots.txt directives. It crawls at a moderate rate—typically 1–5 requests per second—and uses a configurable crawl delay that can be set via the Crawl-Delay directive in robots.txt. The bot originates from a set of public IP ranges listed in Atlassian’s official documentation, including prefixes such as 104.192.136.0/21 and 18.200.0.0/16. It does not follow redirect chains longer than 4 hops and ignores noindex meta tags. The crawler operates only on HTTP and HTTPS protocols, and it respects the If-Modified-Since header to avoid re-fetching unchanged resources. Atlassian provides a dedicated status page (https://developer.atlassian.com/cloud/confluence/crawler/) where site owners can verify the bot’s IP ranges and request exclusion.

📋 robots.txt Compliance

According to Atlassian’s official documentation, atlassian-bot fully honors Disallow directives in robots.txt, including wildcard patterns and path-specific exclusions. The bot also supports the Crawl-Delay directive, allowing webmasters to specify a minimum interval between successive requests. Atlassian explicitly states that the bot “will not crawl any URL that is blocked by robots.txt” and encourages site owners to use standard mechanisms to control access. There is no evidence of non-compliance in public record.

🔍 Detection Indicators

The primary User-Agent token is Atlassian-Bot, with a full string that may include version numbers (e.g., Atlassian-Bot/1.0). Additional identifiers include the User-Agent: Atlassian-Crawler variant used in older implementations. The bot sends a From header containing [email protected] for feedback, and it does not set any custom X-Forwarded-For headers. Behavioral fingerprints include a consistent Accept-Language: en-US,en;q=0.5 header and a request interval that aligns with the configured crawl delay. Security researchers at Shodan and Censys have documented these patterns in their threat intelligence feeds.

📊 Data Usage

Collected data is used to populate Atlassian’s internal search engine (e.g., Confluence Search, Jira Service Management Knowledge Base) and to train Atlassian’s AI features, such as Atlassian Intelligence, which provides automated summaries and suggested responses. The bot does not store personal identifiable information (PII) intentionally, and Atlassian’s privacy policy (https://www.atlassian.com/legal/privacy-policy) states that crawled data is anonymized before being used for model training. The company publishes a transparency report detailing data collection practices.

⚙️ Rate Limiting Policy

Because atlassian-bot can send multiple concurrent requests during initial indexing of a domain, it is rate-limited in production environments to prevent resource exhaustion. The recommended threshold is 1 request per 2 seconds per IP, and blocking is triggered only when the bot exceeds 10 requests per second over a 60-second window—a policy based on standard crawler etiquette and documented in Atlassian’s operational guidelines.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.