How can I see the data content (HTML, JS, etc) in HTTPS responses in Wireshark?

Question

I'm learning packet sniffing and analysis with Wireshark.

At first, I could only see DNS, TCP, and ARP packets. I learned I couldn't see any HTTP packets because it was all encrypted in the transport layer. So I properly set up the env variable $SSLKEYLOGFILE so the browser would now log SSL keys in a given file. After I configured Wireshark to use the keys from the file, I can now view HTTP traffic. However, it seems most (if not all) application data such as HTML, JS and user data is also encrypted.

My questions then are:

Why is data encrypted at transport layer and at application layer? How can I view HTTPS traffic unencrypted? What use is any MITM attack if you can very rarely see data unencrypted? How would one get the keys?

Answer 1

Man-in-the-Middle means that you also control the keys. Encryption means that you cannot "just sniff", and that you need to be "in the middle" to be able to see anything.

How to get the keys as a MITM is straightforward: you are the server the client negotiates TLS with, so you get the key. You pass on the traffic to the intended destination from then on after you negotiate TLS with that server.

What you are likely seeing at the application layer is not encryption but compression and encoding. Wireshark offers tools to decode those things.