10

I have to store pre-master keys for all TLS connections due to regulator request. We are using nginx to terminate TLS.

I've read Extract pre-master keys from an OpenSSL application and especially excellent walkthrough Extracting openssl pre-master secret from apache2 and I was able to record keys from apache, but still have no success with nginx. Perhaps I'm missing something obvious?

I'm using Debian 9 system for testing. First I check which version of openssl is used by nginx:

# nginx -V 2>&1 |grep SSL
built with OpenSSL 1.1.0k  28 May 2019
# ldd /usr/sbin/nginx |grep ssl
    libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007f9bd9f95000)

So I installed libssl-dev package which provides 1.1.0k-1~deb9u1 version.

I downloaded sslkeylog.c and compiled it: cc sslkeylog.c -shared -o libsslkeylog.so -fPIC -ldl

I put compiled library into /usr/local/lib/libsslkeylog.so path for convenience. Then I edited nginx's systemd config:

#cat /etc/systemd/system/nginx.service.d/override.conf
[Service]
Environment=SSLKEYLOGFILE=/tmp/premaster.txt
Environment=LD_PRELOAD=/usr/local/lib/libsslkeylog.so

# systemctl daemon-reload
# systemctl restart nginx

I can see that LD_PRELOAD works - lsof of both master and worker nginx processes show that /usr/local/lib/libsslkeylog.so is loaded:

# lsof -n -p 10313 |grep ssl
nginx   10313 root  mem    REG              254,1   442984   3255 /usr/lib/x86_64-linux-gnu/libssl.so.1.1
nginx   10313 root  mem    REG              254,1    14224  20914 /usr/local/lib/libsslkeylog.so
# lsof -n -p 10314 |grep ssl
nginx   10314 www-data  mem       REG              254,1   442984   3255 /usr/lib/x86_64-linux-gnu/libssl.so.1.1
nginx   10314 www-data  mem       REG              254,1    14224  20914 /usr/local/lib/libsslkeylog.so

But after I access nginx through curl

curl -k -I https://localhost

or through browser on my PC, /tmp/premaster.txt is not created

What am I doing wrong? Or is there a better way to store pre-master keys in nginx?

3
  • 1
    Why would the regulator need that you store all pre-master keys? Do they intend to intercept your communications? I am not convinced he is doing a proper job...
    – Ángel
    Aug 28, 2019 at 1:37
  • Perhaps I was not entirely clear - the regulator is a goverment agency, they have rights for this kind of information according to federal law. Internet Service Provider should supply traffic dump to them and site owner should provide ssl keys so they can decrypt the dump. Aug 28, 2019 at 8:14
  • With sslkeylog.c I've observed that if you have more that one server listening on *:443, secrets are only saved for the first one. May 9, 2020 at 2:20

2 Answers 2

8

Well, as I was pointed at nginx mailing list, nginx removes all environment variables inherited from its parent process except the TZ variable, so once I defined needed variables in nginx.conf

env LD_PRELOAD=/usr/local/lib/libsslkeylog.so;
env SSLKEYLOGFILE=/tmp/premaster.txt;

keys started being recorded as expected.

3
  • Would you know how to perform this on Windows?
    – Dandré
    Feb 27, 2021 at 20:07
  • How did you combine requests in nginx logs and entries in a file premaster.txt?
    – undying
    Jul 28, 2022 at 14:46
  • @user126093, if i remember correctly you'll need traffic dump for that. You can decrypt traffic dump using premaster.txt and then you'll see ip addresses and http requests which can be traced back to nginx log. Jul 29, 2022 at 10:33
2

The answer above seems to be incorrect. nginx doesn't execve() its children, those are just forks of the parent process, so setenv() of LD_PRELOAD doesn't have any effect, it needs to be set before nginx is executed at all, i.e. in SystemD.

However SSLKEYLOGFILE does need to be passed to workers.

So, my advice is to follow the same steps as for Apache, but with the added step of putting env SSLKEYLOGFILE; in nginx.conf.

Summary of the steps needed:

  • compile libsslkeylog.so as usual:

    sudo apt install git make gcc libssl-dev
    git clone --depth=1 https://git.lekensteyn.nl/peter/wireshark-notes
    cd wireshark-notes/src
    make
    sudo install libsslkeylog.so /usr/local/lib/
    
  • set environment variables in systemd, again as usual:

    cat > /etc/systemd/system/nginx.service.d/override.conf <<EOF
    [Service]
    Environment=SSLKEYLOGFILE=/tmp/premaster.txt
    Environment=LD_PRELOAD=/usr/local/lib/libsslkeylog.so
    EOF
    
  • make sure SSLKEYLOGFILE remains accessible to workers, by adding the following to nginx.conf:

    env SSLKEYLOGFILE;
    

Update: It seems this is still not enough for nginx to log all secrets. There's an undocumented OpenSSL function called SSL_set_SSL_CTX which replaces the context of an SSL object. nginx uses this function. libsslkeylog is not prepared for this (it only taps SSL_new and registers keylog on that context). I'll try to contact the author (@lekensteyn) to improve it.

Update 2: If your version of nginx has HTTP/3 support enabled, check whether it was compiled with a fork of OpenSSL that supports HTTP/3, or if it's using the so called "OpenSSL compatibility layer", which is a hack that allows nginx to serve HTTP/3 using vanilla OpenSSL. This hack uses the keylog callback to make it work, so it will conflict with libkeylogssl. To make libkeylogssl work with the hack, some elaborate scheme is needed to enable its keylog callback to work in addition to the one set by the application.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .