locust

Bot User-Agent: locust

🤖 Overview

Locust is an open-source, Python-based load testing tool originally created by Carl Byström and later maintained by the community under the LocustIO organization. It is designed to simulate concurrent user traffic against web applications, APIs, or other systems to measure performance and identify bottlenecks. Unlike traditional crawlers, Locust is a legitimate automated agent used exclusively for performance engineering and capacity planning. The tool feeds collected metrics—such as response times, throughput, and error rates—into dashboards or external analysis pipelines for engineering teams.

🌐 Technical Behavior

Locust executes user-defined Python scripts that define HTTP requests, response validation, and wait times between actions. Under the hood, it leverages gevent for asynchronous, non-blocking I/O, enabling a single process to spawn thousands of concurrent virtual users. The tool supports distributed load generation using a master/worker architecture, where workers run on separate machines and report results to the master via WebSocket. Each virtual user behaves independently, with configurable spawn rate and peak concurrency. By default, Locust uses the Python Requests library for HTTP calls, inheriting its User-Agent string unless overridden. IP ranges for distributed tests are entirely dependent on the deployment environment (e.g., cloud VMs, on-premises servers); there is no fixed IP range associated with the tool itself.

📋 robots.txt Compliance

Locust is not a web crawler and does not automatically parse or obey robots.txt directives. Its primary use case is load testing on infrastructure owned or explicitly authorized by the tester. The official documentation advises users to only target systems they own or have written permission to test. When used improperly against third-party services, Locust disregards robots.txt by design, as it is not engineered for content discovery or indexing.

🔍 Detection Indicators

The default User-Agent string sent by Locust is python-requests/2.x.x (where x matches the installed version of the Requests library), but this is trivially configurable in the test script via the client.headers method. Behavioral fingerprints include: a rapid ramp-up of concurrent connections from a single or small set of source IPs, exact repetition of request patterns, and the absence of browser-like assets (CSS, images, JavaScript). The tool can also send custom HTTP headers defined by the user, such as X-Locust-Test: true but this is not standard.

📊 Data Usage

Locust collects performance metrics including average response time, 90th/99th percentile response times, requests per second, failure rate, and number of concurrent users. This data is streamed in real-time to the web-based UI or exported to CSV/JSON files for later analysis. It does not store or transmit actual response content for AI training or indexing; the tool focuses solely on latency and throughput statistics to help engineers optimize system performance.

⚙️ Rate Limiting Policy

Locust is designed to generate high request rates that can easily overwhelm target servers if left uncontrolled. Rate limiting is essential because the tool’s purpose is to stress-test application limits; threshold-based blocking (e.g., returning HTTP 429 or dropping connections when request rates exceed a defined safe ceiling) protects production environments from unintended degradation while still allowing legitimate load tests against dedicated staging or performance testing environments.

🛡️

Stop Bots. Save Bandwidth. Protect Revenue.

Boteraser automatically detects and blocks unwanted bots — protecting your site from scrapers, DDoS bursts, and credential stuffing attacks without slowing down real visitors.

✅ Start Free Protection

Setup takes under a minute  ·  Free trial available

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.