🤖 Overview

The FrontPage crawler, officially known as the Microsoft FrontPage Server Extensions (FPSE) Client, is a legitimate automated agent operated by Microsoft Corporation. Its primary purpose is to facilitate remote authoring, content management, and web publishing for sites using FrontPage Server Extensions, a technology introduced in the late 1990s and embedded in tools like Microsoft FrontPage and SharePoint Designer. The bot communicates with the server to perform file transfers, link verification, and web administration tasks, feeding data into the authoring client’s local project files.

🌐 Technical Behavior

The FrontPage agent does not perform typical web crawling or indexing; instead, it issues a specific sequence of HTTP methods — including PROPFIND, MKCOL, MOVE, PUT, and DELETE — under the WebDAV protocol. These requests are directed at the _vti_bin virtual directory and other FPSE-specific endpoints. The agent’s request frequency can be high during publication of a large site, as it sends batched requests to upload or synchronize multiple files. It originates from a wide range of IP addresses, typically assigned to the user’s machine or a corporate network, and uses the HTTP 1.1 protocol. According to Microsoft’s official documentation (archived at msdn.microsoft.com), the agent sends a custom header Front-End-Https: On for secure connections.

📋 robots.txt Compliance

The FrontPage agent does not inherently read or honor robots.txt files, as it is not a web crawler but an authoring client operating under user authentication. However, Microsoft’s documentation recommends that site administrators block the _vti_bin directory in robots.txt to prevent search engines from indexing FPSE paths. The agent itself ignores Disallow directives because its traffic is user-initiated, not automated for search indexing.

🔍 Detection Indicators

The most reliable detection indicator is the User-Agent string: Mozilla/4.0 (compatible; MSIE; Windows NT; MS FrontPage 4.0) or variations like MS FrontPage 5.0 depending on the version. Behavioral fingerprints include repeated PROPFIND and PUT requests to /_vti_bin/ paths, distinctive HTTP methods not used by standard browsers, and the presence of the Translate: f header in some requests. Microsoft’s Knowledge Base article Q306798 confirms these identifiers.

📊 Data Usage

Data collected by the FrontPage agent is used exclusively for local authoring and publishing operations. The bot retrieves remote file metadata (such as last-modified timestamps) via PROPFIND to synchronize with local copies, and uploads edited files via PUT. No data is harvested for AI training, analytics, or search indexing — all transfers are user-initiated and directly tied to content management workflows.

⚙️ Rate Limiting Policy

Rate limiting for the FrontPage agent is advisable because during bulk publishing, it can generate a high volume of sequential WebDAV requests that stress server resources. A threshold-based blocking policy (e.g., limiting requests per second from a single IP) protects against accidental denial-of-service while still allowing legitimate authoring sessions to complete.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.