I am working with a website that sends API requests. I would like to write a client to make the requests myself, but in order to do so I would need to first see the request payloads. However, the connection is secured and therefore I can't see the data in wireshark just like that.

I found out that Wireshark supports SSL decryption: http://wiki.wireshark.org/SSL

However, it doesn't explain how the private key file should be obtained or generated, so I went and looked around and found this blog entry: http://blog.stalkr.net/2010/03/codegate-decrypting-https-ssl-rsa-768.html

After finding the appropriate packets as shown in the images, I exported the certificate, but unlike the challenge they were doing, the connection established for me uses RSA-2048 and does not provide the factorization (I'm assuming real certificates do not provide that, only the ones for games and such).

Is it possible for me to decrypt HTTPS packets that are sent to 3rd party websites? How would I generate the key file required?

  • 4
    As an alternative to decrypting the HTTPS traffic, you could potentially use a local proxy such as Fiddler or Burp Suite to achieve the same goal. Essentially you add a man-in-the-middle in your HTTPS connection. – itscooper Aug 6 '14 at 23:31
  • That sounds like an interesting technique to know. I'll check it out. – That Umbrella Guy Aug 7 '14 at 1:04
  • 1
    If you only need to monitor your browser's requests you can use its built-in developer tools or an add-on such as Firebug to look at the requests. No SSL decryption required. – user42178 Aug 14 '14 at 11:38
up vote 11 down vote accepted

Wireshark is a very powerful tool. In most cases, the (addon-less) debug consoles of the browsers firefox and chrome should be enough. Both have network monitors that are sufficient most time. Be aware that the firefox' monitor doesn't support websockets yet.

If you still wanted to use wireshark, then consider utilizing the SSLKEYLOGFILE file, more help at this and this, and your linked wireshark wiki page.

  • 1
    Currently I am using chrome inspector, which sort of does the job, but the service sends binary-serialized data which chrome does not display properly, so I'm unable to reproduce the request bodies myself. I only thought of using wireshark because I've worked with the same protocol before but the connection was not secured and so it was easy to just read it off wireshark. – That Umbrella Guy Aug 7 '14 at 1:00
  • OK, then you are not one of the "most cases" :-) – user10008 Aug 7 '14 at 18:57
  • 2
    I've selected this as the accepted answer because supplying the pre-master key list files obtained via SSLKEYLOGFILE variable did the trick. – That Umbrella Guy Aug 7 '14 at 20:49

The private key is private to the webserver. If you don't control the webserver you shouldn't be able to obtain it. The certificate only holds the public key so it wouldn't be of much use to you. You could try to setup a proxy https server and do a man-in-the-middle attack - in that case you would have the key of your proxy server.

Are you trying to crack the protocol from a software or you are normally using a browser to access the service?

In the second case you can use the temporary for the session by configuring Firefox to log the key to a file, as it was suggested by user10008: look for SSLKEYLOGFILE in the wiki page you linked, and the links that user10008 posted.

  • yes, currently I am accessing it through a browser, but it would be save so much more time if I could just simulate the API calls. – That Umbrella Guy Aug 7 '14 at 0:58
  • your best chance is to try and observe the API calls through the javascript debug tools, instead of trying to use wireshark or similar. Which browser are you using? – pqnet Aug 7 '14 at 1:00
  • Chrome. The API calls are simple enough, except it is binary-serialized, which the inspector cannot render properly. Do you know if there are any extensions that might be able to display request payloads as raw bytes? Then that would be straightforward to work with. – That Umbrella Guy Aug 7 '14 at 1:02
  • 1
    @MxyL There are two kinds of "private keys": one connection-private known by your browser and the server only used for symmetric encryption, and another private certificate key only known by the server, used for assymetric encryption. For the first, you are a "friend" by opening the connection, the second you are none. No one should be, as then that person could read (or, with PFS, MITM) all HTTPS traffic the guy has. The SSLKEYLOGFILE transports the connection-private key from your browser to wireshark. – user10008 Aug 7 '14 at 19:05
  • 1
    @MxyL Nothing to worry, you always get the content. Both keys are used on every connection, this is called hybrid encryption. This is part of TLS, see the second sentence of the wikipedia page, and thats why. – user10008 Aug 8 '14 at 17:10

You can't, unless you have administrative control over the 3rd party web server, or retrieve the certificate via some other nefarious means. SSL/TLS is reliant upon the private certificate staying private.

Furthermore, even if you had the server's private key, you might not be able to decrypt traffic from an earlier session if Perfect Forward Secrecy was used. In that case you would have to know the specific private key used for your single session.

In principle, because you are the client, you are privy to the pre_master_secret which is what you need to derive the master_secret. The master_secret is the symmetric key that's actually used to encrypt your session. As @pqnet said, there are ways to utilise this in FireFox and Chrome, although it is dependant upon the client software, and I'm not sure about Flash. However, there are more trivial methods to inspect your own HTTPS traffic...

Local Proxies

Since it is your own connection, there is no reason why you can't pass the website through a proxy server. Many proxy servers are configured to allow SSL-pass-through, which still gives you end-to-end encryption, but you can break this by terminating your connection at the proxy server (if you trust the proxy's SSL certificate). The proxy will then establish it's own SSL connection to the 3rd party website, passing along any traffic you send. Companies sometimes use the same method to inspect their users' outgoing HTTPS traffic (that's a contentious one). Essentially it's a man-in-the-middle.

Browser <---> Local Proxy <---> Website
          ^                 ^
        HTTPS             HTTPS

Local proxies run on your own machine, and allow you to inspect and even modify traffic that passes through them. If an attacker tried to do this, the user would see a browser warning to show that it's not received a valid certificate for the requested website. However, you can setup your machine to trust the local proxy's CA certificate.

Two popular local proxies that work with HTTPS traffic are:

Burp Suite has AMF serialisation support built-in. Fiddler looks like it has an extension.

  • The master secret is not the symmetric key used to encrypt messages. That's the session key ('write key' in RFC parlance). – user207421 Mar 21 '17 at 4:29

SSL was built to prevent you from doing exactly what you're attempting. You need the key to get access to private communications, with or without Wireshark.

Several answers suggest a man-in-the-middle attack, which should work with a lot of effort... but if this is a public API, it's probably documented. Particularly, if it is a web service API, you can get the WSDL from the browser by appending the endpoint URL with ?WSDL. This will give a complete list of methods and return types.

Adding to itscooper's message, you can also use Charles Proxy with a trusted certificate installed on the device/browser and allow Charles to decrypt SSL so you can read the traffic. I do this quite a lot when testing devices that are communicating over SSL.

http://www.charlesproxy.com/documentation/proxying/ssl-proxying/

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.