By now everyone knows about the OpenSSL Heartbleed vulnerability: a missing bounds check in one of the most popular TLS implementations has made millions of Web servers (and more) leak all sorts of sensitive information from memory. This can leak login credentials, authentication cookies, and Web traffic to attackers. But could it be used to recover the site’s TLS private key? This would enable complete decryption of previously-recorded traffic if perfect forward secrecy was not negotiated at the time and otherwise Man-in-The-Middle attacks to all future TLS sessions.
Since this would be a much more serious consequence of Heartbleed, I decided to investigate. The results were positive: I was able to extract private keys from a test Nginx server after a few days’ work. Later I applied my techniques to solve the CloudFlare Challenge. Along with a few other security researchers, we independently demonstrated that RSA private keys are indeed at risk. Let’s go through the details on how to extract the private key and why the attack is possible.
How to extract the private key
Readers not familiar with RSA can read about it here. To simplify things a bit, a large (2048 bits) number N is constructed by multiplying together two large randomly generated prime numbers p and q. N is made public while p and q are kept secret. Finding p or q allows recovery of the private key. A generic attack is just factorizing N, but this is believed to be difficult. However, with a vulnerability like Heartbleed, the attack is much simpler: since the Web server needs the private key in memory to sign the TLS handshake, p and q must live in memory and we can try to obtain them with Heartbleed packets. The problem simply becomes how to identify them in the returned data. This is easy, as we know p and q are 1024 bits (128 bytes) long, and OpenSSL represents big numbers little-endian in memory. A brute-force approach treating every 128 consecutive bytes in the Heartbleed packets as little-endian numbers and testing if it divides N is sufficient to spot potential leaks. This is how most people solved the CloudFlare challenge.
But wait, isn’t our scenario just like a cold boot attack? There has been a lot of research on recovering RSA private keys with partial information. One of the most famous papers fromCoppersmith presents message recovery attacks on related messages or insufficient padding, as well as factorization with partial knowledge with the help of lattice basis reduction. With the Coppersmith attack, N can be efficiently factorized if either the top or bottom half bits of p is known. Put it in context, we only need the top/bottom 64 bytes of p in order to compute the private keys, compared to the naive brute-force which requires all 128 bytes. In practice, reaching Coppersmith’s limit is computationally expensive (although still much better than factorization), but assuming 77 bytes (60 percent) known we can comb the Heartbleed packets for potential private key segments very quickly.
In retrospect, there were 242 instances of private key remnants suitable for the Coppersmith attack out of 10,000 packets (64 KB each) that I collected. Implementation of the Coppersmith attack was made easy thanks to the comprehensive computer algebra building blocks from Sage (although I later found out that Sage already implements Coppersmith).
Can we do even better? If you have ever viewed a RSA private key using openssl rsa -text -in server.key, you will notice that there are quite a few numbers other than the two prime factors p andq. In fact, they are precomputed values for RSA’s Chinese Remainder Theorem optimization. If some of them are leaked, they can also be used to deduce p. And what about the Montgomery representations of p and q that OpenSSL uses for fast multiplication? They also admit a variant of Coppersmith so partial bits are useful as well. With this in mind, I set out to search for them in the collected packets from my test server where these values are all known. But not even single one of them appears partially (> 16 bytes) in the dataset. How is this possible?
Note: all my experiments and the CloudFlare challenge are targeting Nginx which is single-threaded. It is possible that for a multi-threaded wWb server, more leakage can be observed.
Why p and only p leaks
When Heartbleed first came out, people argued that RSA private keys should not be leaked. After all, they are loaded when the Web server starts, so they occupy a lower memory address. And as the heap grows upwards, a later-allocated buffer leaked by Heartbleed shouldn’t reach them. This argument is consistent with my inability to find any CRT precomputed values with one exception: p is definitely leaked somehow. If we assume this argument is correct, the question becomes: why is pleaked?
To add more to the mystery, OpenSSL apparently scrubs all temporary BigNums it used. To reduce the overhead of dynamically allocating temporary values, OpenSSL provides a BN_CTX which is a pool of BigNums operating in stack (LIFO) fashion. Upon finishing, the context is destroyed and all allocated buffers are scrubbed. This would mean that when the Heartbleed packet is constructed there shouldn’t be any temporary values left in memory (again assuming single thread) because the BN_CTX has long been released.
I won’t bother you with all the pain I went through to identify the cause, so here is the spoiler: when a BigNum is being extended to a bigger buffer size, its original buffer is not zeroized before being freed. The chain of the control flow path that leads to the leak of p is even subtler. During initial TLS handshake, the server key exchange is signed with the private key. The CRT signing performs amodulo p operation, which caused p<<BN_BITS2 to be stored in a temporary variable allocated from the BN_CTX pool. Later in the CRT fault-injection check, that temporary variable is reused (remember BN_CTX operates like a stack) as val. An interesting fact is that a reallocated temporary variable only gets its lowest nibble zeroized, and in the case of p<<BN_BITS2 nothing is destroyed. valimmediately receives a Montgomery-reduced value, but since the original buffer is unable to accommodate the new value, it gets extended and p gets released to free heap space, waiting to be grabbed. And because this happens every time a TLS handshake occurs, it can be spilled everywhere.
Since it is difficult to find which BigNums may be extended and leaked statically, I instrumented OpenSSL and experimented a bit. It turned out that a shifted version of the Montgomery representation of p will also be freed at the leaky point, but that only happened at the first RSA exponentiation when the Montgomery context is initialized. It will live in a low memory address and I was unable to locate it in any collected packets.
The OpenSSL team has been notified about the above leak. Although to be fair, this is not exactly a security bug by itself as OpenSSL is never explicitly designed to prevent sensitive data leakage to heap.
Rubin Xu is a PhD candidate at Cambridge University, in the security group of the Computer Laboratory. His thesis is on mobile security although he also has an interest in cryptography. He was one of four people to successfully win a public challenge to exploit the catastrophic Heartbleed vulnerability to steal private cryptography keys that constitute a website’s crown jewels. This post wasfirst published on the Light Blue Touchpaper blog. Xu would like to thank Joseph Bonneau for the useful comments and proof reading.