In-Depth

Updated OpenSSL Libraries Patch Potential Information Disclosure, DoS Flaws

SSL and TSL are used to encrypt almost all sensitive communications traffic on the Internet.

• By Stephen Swoyer
• 01/30/2012

Although the Windows world is focused on Microsoft Corp.'s January, 2012 "Patch Tuesday" release of security fixes, information security professionals are likely focused on something else: the recent release of updated libraries for OpenSSL, a widely-used open source implementation of the Secure Sockets Layer (SSL) and Transport Layer Security (TLS) encryption protocols.

SSL and TSL are used to encrypt almost all sensitive communications traffic on the Internet. The OpenSSL library is the most widely used open source implementation of both standards. What's more, free or open source software platforms -- chiefly, the so-called LAMP (Linux, Apache, MySQL, Python) stack -- power a disproportionate share of the Internet's SSL-dependent resources.

That's why the OpenSSL refresh is a Very Big Deal.

The latest OpenSSL library releases, dubbed 1.0.0f and 0.9.8s, fix a total of six issues. The most prominent of these is a flaw in OpenSSL versions 1.0.0 and 0.9.8 that has to do with the handling of datagram transport layer security (DTLS) communications. An attacker who successfully exploits this flaw could recover encrypted DTLS traffic as plain text, regardless of the cipher strength.

The DLTS vulnerability was discovered by security researchers Nadhem Alfarden and Kenny Paterson, from Royal Holloway, University of London (RHUL). It's actually a variant of an existing attack -- also developed at RHUL -- that involves the ISO Cipher Block Chaining (CBC) mode encryption standard.

Both Alfarden and Paterson plan to present details about the OpenSSL DLTS attack at next month's Network and Distributed System Security (NDSS) Symposium in San Diego. The new libraries will give admins a chance to patch their OpenSSL implementations well ahead of that disclosure.

Another significant vulnerability is an SSL padding issue that could permit the encrypted contents of memory that has previously been freed -- albeit only 15 bytes-worth -- to be sent to SSL peers. "This affects both clients and servers that accept SSL 3.0 handshakes: those that call SSL_CTX_new with SSLv3_{server|client}_method or SSLv23_{server|client}_method. It does not affect TLS," the OpenSSL security advisory says. The vulnerability is mitigated to a large degree because most OpenSSL deployments use only a single write buffer per connection. "This, combined with the small number of bytes leaked per record, serves to limit [the] severity of this issue," the advisory concludes.

The OpenSSL 0.9.8s release patches a flaw specific to OpenSSL 0.9.8 that can result in a so-called "double free error," which in turn opens it up to a buffer overflow attack. This flaw, which isn't present in OpenSSL v1.0.0, only occurs when the X509_V_FLAG_POLICY_CHECK flag is set.

A flaw that results in malformed RFC 3779 data affects OpenSSL versions 1.0.0 and 0.9.8. If malformed RFC 3779 data is included in an X509 certificate, it could trigger an assertion failure that might, in turn, be exploited by a denial of service (DoS) attack. This attack, too, is mitigated, according to the OpenSSL security advisory: "[I]n the standard release of OpenSSL, RFC 3779 support is disabled by default, and in this case OpenSSL is not vulnerable. Builds of OpenSSL are vulnerable if configured with 'enable-rfc3779'."

Another DoS attack exploits a flaw in OpenSSL's support for handshake restarts in the context of server gated cryptography (SGC). It affects both OpenSSL libraries. Ditto for a DoS attack involving gosudarstvennyy standart (GOST), a set of ISO-like standards developed in the former Soviet Union and currently administered by the Commonwealth of Independent States. "A malicious TLS client can send an invalid set of GOST parameters which will cause the server to crash due to lack of error checking," the advisory indicates.