🤖 Overview

isitwp.com is a web crawler operated by IsItWP, a service provided by the same team behind WPBeginner and other WordPress-focused platforms. Its primary purpose is to scan websites to determine whether they are powered by WordPress, collecting metadata such as theme and plugin usage for a publicly accessible directory of WordPress sites. The data feeds into the IsItWP website (isitwp.com) where users can look up any domain to see if it runs WordPress and what specific technologies it employs.

🌐 Technical Behavior

The isitwp.com bot performs an initial HTTP GET request to a site’s root domain, followed by probes to detect common WordPress files such as /wp-admin/, /wp-content/, /wp-includes/, and /xmlrpc.php. It also checks for typical WordPress meta tags and generator headers. The crawler typically sends requests at intervals of several seconds to avoid overwhelming servers, but during scheduled bulk scans it may sustain a higher rate. Documented IP ranges are not publicly listed, but the bot originates from IPs belonging to cloud hosting providers used by the IsItWP infrastructure. It uses standard HTTP/1.1 and HTTPS, and does not appear to employ JavaScript rendering or headless browsers.

📋 robots.txt Compliance

According to publicly available documentation on the IsItWP website, the isitwp.com bot honors robots.txt Disallow directives. Site owners can block the bot entirely by adding a Disallow: / rule for the user‑agent IsItWP or isitwp.com. The IsitWP team explicitly states that they respect robots.txt and will not crawl disallowed paths.

🔍 Detection Indicators

The primary User‑Agent string reported in server logs is IsItWP (e.g., Mozilla/5.0 (compatible; IsItWP; +https://isitwp.com)) or simply isitwp.com. Additional identifying headers include a non‑empty From: value or a custom X-Source: isitwp header. The bot does not spoof common browser user‑agents, making it straightforward to identify in access logs. Requests for /wp-admin/admin-ajax.php and /wp-json/ are also typical fingerprints.

📊 Data Usage

Collected data is used to populate the public database on isitwp.com, allowing anyone to look up a domain and see whether it uses WordPress, along with detected themes, plugins, and version information. The aggregated data is also used for analytics and lead generation by the WPBeginner ecosystem, but is not sold to third parties for direct marketing. The service explicitly states that it does not store personal data from crawled sites.

⚙️ Rate Limiting Policy

While the isitwp.com bot is legitimate, it can send a high volume of requests during large‑scale scans of the Internet. Rate limiting is recommended because sustained aggressive crawling may degrade web server performance. Administrators should implement threshold‑based blocking for any single IP that issues more than, for example, 10 requests per minute, while allowing the User‑Agent to be whitelisted if desired.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.