Extract pre-master keys from an OpenSSL application

Question

Consider an application using OpenSSL which has a bug. A packet capture of the full SSL session is available, as well as a core dump and debugging symbols for the application and libraries. A RSA private key is also available, but since a DHE cipher suite is in use, this cannot be used to decrypt the packet capture using Wireshark.

Thomas suggests in this post that it is possible to extract keys from RAM. How could this be done for OpenSSL? Assume that the address of the SSL data structure is known and TLS 1.0 is in use.

Answer 1

Note: as of OpenSSL 1.1.1 (unreleased), it will be possible to set a callback function that receives the key log lines. See the SSL_CTX_set_keylog_callback(3) manual for details. This can be injected as usual using a debugger or a LD_PRELOAD hook. Read on if you are stuck with an older OpenSSL version.

For a walkthrough of the LD_PRELOAD approach for Apache on Debian Stretch, see my post to Extracting openssl pre-master secret from apache2. Technical details follow below.

---

If you have just gdb access to the live process or a core dump, you could read data from data structures. It is also possible to use an interposing library.

In the following text, the basic idea of key extraction using GDB is described, then an automated script is given to perform the capture.

Using GDB (basic idea)

Based on this Stackoverflow post, I was able to construct a function that could print a line suitable for Wireshark's key logfile. This is especially useful while analyzing core dumps. In GDB, execute:

python
def read_as_hex(name, size):
    addr = gdb.parse_and_eval(name).address
    data = gdb.selected_inferior().read_memory(addr, size)
    return ''.join('%02X' % ord(x) for x in data)

def pm(ssl='s'):
    mk = read_as_hex('%s->session->master_key' % ssl, 48)
    cr = read_as_hex('%s->s3->client_random' % ssl, 32)
    print('CLIENT_RANDOM %s %s' % (cr, mk))
end

Then later on, after you step upwards in the stack until you get a SSL structure, invoke the python pm() command. Example:

(gdb) bt
#0  0x00007fba7d3623bd in read () at ../sysdeps/unix/syscall-template.S:81
#1  0x00007fba7b40572b in read (__nbytes=5, __buf=0x7fba5006cbc3, __fd=<optimized out>) at /usr/include/x86_64-linux-gnu/bits/unistd.h:44
#2  sock_read (b=0x7fba60191600, out=0x7fba5006cbc3 "\027\003\001\001\220T", outl=5) at bss_sock.c:142
#3  0x00007fba7b40374b in BIO_read (b=0x7fba60191600, out=0x7fba5006cbc3, outl=5) at bio_lib.c:212
#4  0x00007fba7b721a34 in ssl3_read_n (s=0x7fba60010a60, n=5, max=5, extend=<optimized out>) at s3_pkt.c:240
#5  0x00007fba7b722bf5 in ssl3_get_record (s=0x7fba60010a60) at s3_pkt.c:507
#6  ssl3_read_bytes (s=0x7fba60010a60, type=23, buf=0x7fba5c024e00 "Z", len=16384, peek=0) at s3_pkt.c:1011
#7  0x00007fba7b720054 in ssl3_read_internal (s=0x7fba60010a60, buf=0x7fba5c024e00, len=16384, peek=0) at s3_lib.c:4247
...
(gdb) frame
#4  0x00007fba7b721a34 in ssl3_read_n (s=0x7fba60010a60, n=5, max=5, extend=<optimized out>) at s3_pkt.c:240
240     in s3_pkt.c
(gdb) python pm()
CLIENT_RANDOM 9E7EFAC51DBFFF84FCB9...81796EBEA5B15E75FF71EBE 6ED2EA80181...

Note: do not forget to install OpenSSL with debugging symbols! On Debian derivatives it would be named something like libssl1.0.0-dbg, Fedora/RHEL call it openssl-debuginfo, etc.

Using GDB (improved, automated approach)

The basic idea which is described above works for small, manual tests. For bulk extraction of keys (from a SSL server for example), it would be nicer to automate extraction of these keys.

This is done by this Python script for GDB: https://git.lekensteyn.nl/peter/wireshark-notes/tree/src/sslkeylog.py (see its headers for installation and usage instructions). It basically works like this:

• Install breakpoints on several functions where new pre-master keys can arise.
• Wait for the function to finish and write these keys (if not known before) to a file (which follows the SSLKEYLOGFILE format from NSS).

Paired with Wireshark, you perform a live capture from a remote server by running these commands:

# Start logging SSL keys to file premaster.txt. Be careful *not* to
# press Ctrl-C in gdb, these are passed to the application. Use
# kill -TERM $PID_OF_GDB (or -9 instead of -TERM if that did not work).
(server) SSLKEYLOGFILE=premaster.txt gdb -batch -ex skl-batch -p `pidof nginx`
# Read SSL keys from the remote server, flushing after each written line
(local) ssh user@host stdbuf -oL tailf premaster.txt > premaster.txt
# Capture from the remote side and immediately pass the pcap to Wireshark
(local) ssh user@host 'tcpdump -w - -U "tcp port 443"' |
    wireshark -k -i - -o ssl.keylog_file:premaster.txt

Using LD_PRELOAD

SSL/TLS can only negotiate keys at the SSL handshake steps. By interposing the library interfaces of OpenSSL (libssl.so) that performs said actions you will be able to read the pre-master key.

For clients, you need to interpose SSL_connect. For servers you need to interpose SSL_do_handshake or SSL_accept (depending on the application). To support renegotiation, you will also have to intercept SSL_read and SSL_write.

Once these functions are intercepted using a LD_PRELOAD library, you can use dlsym(RTLD_NEXT, "SSL_...") to lookup the "real" symbol from the SSL library. Call this function, extract the keys and pass the return value.

An implementation of this functionality is available at https://git.lekensteyn.nl/peter/wireshark-notes/tree/src/sslkeylog.c.

Note that different OpenSSL versions (1.0.2, 1.1.0, 1.1.1) are all incompatible with each other. If you have multiple OpenSSL versions installed and need to build an older version, you might have to override the header and library paths:

make -B CFLAGS='-I/usr/include/openssl-1.0 -DOPENSSL_SONAME=\"libssl.so.1.0.0\"'

Answer 2

The master secret is in SSL->session->master_key.

Alternatively, you can get the session struct as follows:

SSL_SESSION ss = SSL_get_session(SSL);

A stated above, there is a master_key field in the SSL_SESSION struct.

Or you can print the session details (including the master_secret) using SSL_SESSION_print() or SSL_SESSION_print_fp().

I do not think the pre_master_secret can be retrieved once the master_secret has been computed.

Answer 3

Old thread, but an update for anyone reading it: https://github.com/openssl/openssl/pull/25297#top

Will be merged and available in openssl-3.5, and so the use of SSLKEYLOGFILE should "just work" moving forward

Answer 4

I found mitmproxy was able to generate the needed pre-master SSL key log I needed to inspect traffic created outside of the web browser (in my case, a F5 WAF blocked PATCH request made by docker push).

I ran mitmproxy w/SSLKEYLOGFILE set and then routed docker traffic through the proxy using either proxychains or the HTTPS_PROXY env var. Finally, the pre-master log has the needed entries to decrypt traffic in wireshark.

Though, for my purposes, I no longer needed wireshark to see the response of the failed request -- mitmproxy had that.

SSLKEYLOGFILE="$HOME/premaster.txt" mitmproxy

# for inspecting HTTP, mitmproxy might be sufficient on it's own. otherwise wireshark.
tshark -i wlan0 -o  tls.keylog_file:$HOME/premaster.txt -o tls.debug_file:tls-debug.log  -w captured_requests.pcap

# run commands generating traffic to capture
HTTPS_PROXY=http://localhost:8080 docker push ....

# alternatively, if command doesn't follow HTTPS_PROXY 
proxychains my_program