Junglewise Threat Intelligence

CVE-2023-6237: OpenSSL RSA denial of service in EVP_PKEY_public_check

CVE-2023-6237 · Severity: medium · CVSS 5.9 · Published 2024-04-25

Technologies: OpenSSL Foundation OpenSSL.

Executive brief

OpenSSL is a widely used security library that provides encryption and secure communication for many applications and operating systems. A flaw in how it validates certain RSA security keys can cause the system to spend an excessive amount of time on complex mathematical calculations. An attacker could exploit this by providing a specially crafted, invalid key, causing the application to become unresponsive and leading to a denial-of-service (DoS) condition.

Technical details

A vulnerability exists in OpenSSL's RSA public key validation logic within the EVP_PKEY_public_check() function. When checking a public key, the library performs a primality test (Miller-Rabin) to confirm the RSA modulus is composite. If an attacker supplies an RSA key where the modulus is an extremely large prime number, the computation takes an excessive amount of time because there was previously no limit on the modulus size or the number of Miller-Rabin rounds for this specific check. This can be triggered via the 'pkey' command-line tool or any application using the affected function on untrusted keys. The fix introduces a 16,384-bit limit (OPENSSL_RSA_MAX_MODULUS_BITS) and caps the Miller-Rabin rounds to five.

Affected products

  • OpenSSL Foundation OpenSSL 3.0.0 to 3.0.12, 3.1.0 to 3.1.4, 3.2.0

Timeline

  • 2023-11-02: disclosed: Reported by OSS-Fuzz
  • 2024-01-15: advisory: OpenSSL security advisory published
  • 2024-04-25: other: NVD record published

References