Executive brief
### Summary
When exporting telemetry to a back-end/collector over gRPC or HTTP using OpenTelemetry Protocol format (OTLP), if the request results in a unsuccessful request (i.e. HTTP 4xx or 5xx), the response is read into memory with no upper-bound on the number of bytes consumed.
This could cause memory exhaustion in the consuming application if the configured back-end/collector endpoint is attacker-controlled (or a network attacker can MitM the connection) and an extremely large body is returned by the response.
### Details
https://github.com/open-telemetry/opentelemetry-dotnet/pull/6564 introduced a change to read the response body when a non-200 HTTP status code is received when exporting telemetry to aid debugging by operators so that the error response is included in the logs emitted by the exporter for both [gRPC](https://github.com/open-telemetry/opentelemetry-dotnet/blob/640cf63628567b76b348b26988920dbc0b5c1662/src/OpenTelemetry.Exporter.OpenTelemetryProtocol/Implementation/ExportClient/OtlpGrpcExportClient.cs#L123-L134) and [HTTP/protobuf](https://github.com/open-telemetry/opentelemetry-dotnet/blob/640cf63628567b76b348b26988920dbc0b5c1662/src/OpenTelemetry.Exporter.OpenTelemetryProtocol/Implementation/ExportClient/OtlpHttpExportClient.cs#L36-L41).
An unintended consequence of this change is that the response body is [fully read into memory when received with no upper-bound](https://github.com/open-telemetry/opentelemetry-dotnet/blob/640cf63628567b76b348b26988920dbc0b5c1662/src/OpenTelemetry.Exporter.OpenTelemetryProtocol/Implementation/ExportClient/OtlpExportClient.cs#L68-L89).
This vulnerability was surfaced during the investigation of GHSA-w8rr-5gcm-pp58.
### Impact
If an application using the OTLP exporter is configured to use a back-end/collector endpoint that is attacker-controlled (or a network attacker can MitM the connection) and an extremely large body is returned by the response the application could have its memory exhausted and create a denial-of-service condition.
### Mitigation
The application's configured back-end/collector endpoint needs to behave maliciously. If the collector/back-end is a well-behaved implementation response bodies should not be excessively large if a request error occurs.
### Workarounds
None known.
### Remediation
[#7017](https://github.com/open-telemetry/opentelemetry-dotnet/pull/7017) updates the OTLP exporter for both gRPC and HTTP to:
- Limit the number of bytes read from the response body in an error condition to 4MiB (see https://github.com/open-telemetry/opentelemetry-proto/pull/781); - Only attempt to read the response body if OpenTelemetry error logging is enabled.
Affected products
- NuGet OpenTelemetry.Exporter.OpenTelemetryProtocol
References
- https://api.github.com/users/martincostello
- https://github.com/martincostello
- https://api.github.com/users/martincostello/gists%7B/gist_id%7D
- https://api.github.com/users/martincostello/repos
- https://avatars.githubusercontent.com/u/1439341?v=4
- https://api.github.com/users/martincostello/events%7B/privacy%7D