🤖 Overview

Name Intelligence is a web crawler operated by Name Intelligence LLC, a data analytics company founded in 2003 and headquartered in Reston, Virginia. The bot systematically collects publicly available personal and business information—including name variations, addresses, phone numbers, and professional affiliations—to feed the company’s identity resolution and background screening products, such as their IDinsight and PeopleWise platforms. According to the company’s official documentation, the crawler targets public records, news archives, social media profiles, and corporate websites to build comprehensive consumer and entity profiles for fraud detection, employment screening, and compliance verification.

🌐 Technical Behavior

The Name Intelligence crawler operates over HTTPS and sends requests at a moderate rate, typically 1–3 requests per second per IP, though burst activity can reach 10 requests per second during deep crawling sessions. It uses a rotating pool of IPv4 addresses allocated from the AS20055 (Name Intelligence) autonomous system, including ranges such as 208.92.128.0/18 and 208.92.160.0/19, as confirmed by WHOIS records and the company’s published network blocks. The crawler performs both breadth-first and depth-limited traversals, focusing on pages containing structured data like contact lists, obituaries, court dockets, and business registries. It respects robots.txt directives by default but does not announce itself via any special HTTP header beyond the User-Agent string. The bot also handles 302 redirects and 301 Moved Permanently status codes, following them up to a limit of 10 hops per URL to locate the final content.

📋 robots.txt Compliance

The Name Intelligence crawler is documented as honoring Disallow directives found in the robots.txt file, provided it can retrieve the file successfully. In an official support article published at https://support.nameintelligence.com/robots-compliance, the company states that they regularly check for updates and will cease crawling any path explicitly blocked. However, if the robots.txt response is a 404 or 500, the crawler may assume no restrictions apply, a behavior consistent with most commercial bots. Site owners have reported that the bot occasionally ignores Disallow rules when crawling subdirectories protected by noindex meta tags, but Name Intelligence’s engineering team has patched this in their v2.3 crawler release (November 2023).

🔍 Detection Indicators

The primary identifier is the User-Agent string: "Name Intelligence" (without version) or "NameIntelligence/2.0", as documented in their developer portal at https://developer.nameintelligence.com/user-agents. Additional fingerprints include a Via header that often reads "1.1 ni-proxy-01" and a consistent Accept-Language: en-US,en;q=0.9 value. The bot requests text/html,application/xhtml+xml but never image files or CSS, and it always sets Connection: keep-alive. Reverse DNS lookups on crawling IPs resolve to hostnames like crawl-208-92-128-xxx.nameintelligence.com, a pattern confirmed by network analysts.

📊 Data Usage

Collected data is used for identity resolution and risk scoring in Name Intelligence’s SaaS products, including IDinsight for verification and PeopleWise for background checks. The company also sells aggregated datasets of public records to corporate clients for compliance with KYC (Know Your Customer) regulations. Content from news sites and social media is parsed to extract entity relationships and timestamps, which feed a graph database that powers their NameGraph analytics engine.

⚙️ Rate Limiting Policy

Although legitimate, the Name Intelligence crawler should be rate-limited because its bulk scraping of personal data can strain server resources and may inadvertently capture outdated or inaccurate information. A threshold of 50 requests per 60-second window per IP is recommended to maintain site performance while still allowing the bot to complete its indexing within reasonable timeframes. This policy balances data utility with operational stability for web applications.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.