OpenTelemetry dotnet: OTLP exporter reads unbounded HTTP response bodies
Summary
When exporting telemetry to a back-end/collector over gRPC or HTTP using OpenTelemetry Protocol format (OTLP), if the request results in a unsuccessful request (i.e. HTTP 4xx or 5xx), the response is read into memory with no upper-bound on the number of bytes consumed.
This could cause memory exhaustion in the consuming application if the configured back-end/collector endpoint is attacker-controlled (or a network attacker can MitM the connection) and an extremely large body is returned by the response.
Details
https://github.com/open-telemetry/opentelemetry-dotnet/pull/6564 introduced a change to read the response body when a non-200 HTTP status code is received when exporting telemetry to aid debugging by operators so that the error response is included in the logs emitted by the exporter for both [gRPC](https://github.com/open-telemetry/opentelemetry-dotnet/blob/640cf63628567b76b348b26988920dbc0b5c1662/src/OpenTelemetry.Exporter.OpenTelemetryProtocol/Implementation/ExportClient/OtlpGrpcExportClient.cs#L123-L134) and [HTTP/protobuf](https://github.com/open-telemetry/opentelemetry-dotnet/blob/640cf63628567b76b348b26988920dbc0b5c1662/src/OpenTelemetry.Exporter.OpenTelemetryProtocol/Implementation/ExportClient/OtlpHttpExportClient.cs#L36-L41).
An unintended consequence of this change is that the response body is [fully read into memory when received with no upper-bound](https://github.com/open-telemetry/opentelemetry-dotnet/blob/640cf63628567b76b348b26988920dbc0b5c1662/src/OpenTelemetry.Exporter.OpenTelemetryProtocol/Implementation/ExportClient/OtlpExportClient.cs#L68-L89).
This vulnerability was surfaced during the investigation of GHSA-w8rr-5gcm-pp58.
Impact
If an application using the OTLP exporter is configured to use a back-end/collector endpoint that is attacker-controlled (or a network attacker can MitM the connection) and an extremely large body is returned by the response the application could have its memory exhausted and create a denial-of-service condition.
Mitigation
The application's configured back-end/collector endpoint needs to behave maliciously. If the collector/back-end is a well-behaved implementation response bodies should not be excessively large if a request error occurs.
Workarounds
None known.
Remediation
[#7017](https://github.com/open-telemetry/opentelemetry-dotnet/pull/7017) updates the OTLP exporter for both gRPC and HTTP to:
- Limit the number of bytes read from the response body in an error condition to 4MiB (see https://github.com/open-telemetry/opentelemetry-proto/pull/781);
- Only attempt to read the response body if OpenTelemetry error logging is enabled.
