114

I found out that my ISP does deep packet inspection. Can they see the contents of HTTPS connections? Wouldn't having HTTPS ensure that they can't see the contents being transferred?

And can having a VPN protect me against deep packet inspection by ISPs?

  • 8
    Immediately made me think of this. – einpoklum Mar 28 '17 at 19:53
  • 14
    how do you know your ISP is doing DPI? – SpaceDog Mar 30 '17 at 5:42
  • 15
    @SpaceDog: it is required by law in some countries, which makes it very easy to tell that one's ISP is doing it! – dave Mar 30 '17 at 5:48
  • 3
    @SpaceDog, one of their employees anonymously accepted that they performed DPI to some news outlet. – cppanonhelp666 Mar 30 '17 at 16:56
  • 1
    Yes to VPN!!! All they would see is the IP of your VPN end point! If you are concerned at all and willing to spend money on VPN then do it, main reason I don't is the speed will be effected – FreeSoftwareServers Mar 30 '17 at 18:15
131

Deep Packet Inspection, also known as complete packet inspection, simply means they are analyzing all of your traffic as opposed to just grabbing connection information such as what IP's you are connecting to, what port number, what protocol and possibly a few other details about the network connection.

This is normally discussed in contrast to the gathering of NetFlow information which mainly collects the information listed above.

Deep packet inspection gives your provider a lot of information about your connections and habits of Internet usage. In some cases, the full content of things like SMTP e-mails will be captured.

HTTPS does encrypt the connections but your browser has to make DNS requests which are sent primarily via UDP so that data will be collected as will any unencrypted links or unencrypted cookies sent incorrectly without https. These additional bits which will be collected may be very telling about what type of content you are looking at.

The larger concern for most people is about data aggregation, by collecting this information a data scientist could create a fingerprint for your Internet usage and later associate with past activities or activities from other locations (when you are at work or are on vacation). Likewise, your service provider may choose to sell this to any number of organizations (possibly including criminal organizations) where it could then be used against you in ways. In many countries, people have an expectation that their communications are considered to be private and collecting this data very much goes against that privacy expectation.

Another interesting aspect of this is in the cases like the US where this data may soon be sold it allows International communications sent to people, or servers, in the US be sold as well. Likewise, this could potentially allow every agency from local law-enforcement, military, tax authorities, immigration authorities, politicians, etc. a way to bypass long-standing laws which have prevented them from accessing this type of information, or important informational subsets within this data otherwise.

A slightly different concern when this data can be sold is competitive intelligence / corporate espionage. In the scenario where a company does a lot of research-intensive work at their headquarters located in some small geographical location (think of pharmaceuticals or a defense contractor) selling that data makes it possible for anyone to buy all of the traffic from the local ISP where most of those researchers live and analyze what they are looking for when at home, possibly even directly from the ISP hosting the traffic for their corporate headquarters. If other countries aren't selling similar data it gives foreign companies and companies wise enough to try and buy this data a huge technical advantage. Likewise, it would also allow foreign governments to buy ISP traffic which includes the data from US (or other government) Officials homes.

Imagine companies monitoring their employee's behavior at home or on their mobile devices.

This will likely have a chilling effect on activists and whistle-blowers as well.

Likewise, if credit cards or PII are sent in the clear to a poorly secured remote site your ISP's data set now has a potential PCI or PII regulatory issue on their hands. So this amplifies data-leakage problems of all types by making additional copies of the data leaked.

With the examples I've just mentioned above, and there are hundreds of others, it should be easy to see why this type of data collection has a different level of importance to it than just metadata or basic connection information. Even if your ISP never sells this data they are collecting quite an interesting dataset.

It's a security issue that definitely has a lot of potential long-term security implications.

  • 50
    "In many countries, people have an expectation that their communications are considered to be private and collecting this data very much goes against that privacy expectation." It seems certain members of the US government aren't fond of that, though. – JAB Mar 27 '17 at 23:42
  • 46
    In many EU countries the sale of this data would be against people's privacy expectations AND against the law. Other countries aren't as concerned about privacy. – dave Mar 28 '17 at 2:08
  • 45
    At least in germany, deep packet inspection is illegal. – Magisch Mar 28 '17 at 6:40
  • 10
    Even without the DNS aspect, TLS with SNI (which is what is commonly used today) transmits the hostname in the clear in the initial TLS exchange, before encryption is set up. – a CVn Mar 28 '17 at 12:29
  • 9
    @JohnU: That's unlikely and your claim is a severe disservice. There is no way to "see inside" HTTPS without active attacks that not only inspect the content but change it. Modern client software will not allow this unless it's been backdoored (possible if ISP tricked you into installing software they provided), and savvy users will notice any such active attacks, making them high-risk for the attacker. So for practical purposes, no, nobody is "seeing inside" your HTTPS. – R.. Mar 29 '17 at 18:11
60

Trey Blalock's answer describes percisely what deep packet inspection (DPI) is. But I'd like to add three things to hopefully answer your specific questions:

  1. There is a technique of DPI that does decrypt your data, called SSL interception, although it is more common in enterprise situations and only possible if the ISP (or any other interceptor) has the ability to install a certificate on your machine. So unless the ISP had some way of doing that (technician etc.) this is probably off the table.
  2. HTTPS would prevent the ISP from being able to read data. Of course, This is only true for services that use HTTPS (which unfortunately is not all of them). Also you need to consider that the ISP can read metadata whether the connection is encrypted or not.
  3. A VPN would protect you against DPI performed by the ISP (not by the VPN provider). This is thanks to the fact that VPNs use an encrypted tunnel to connect you to the 'exit node'. This encrypts all of your traffic, and all of the metadata will show packets leaving your computer and going to the VPN server (thus not disclosing the actual server you are accessing).
  • 53
    Note that some ISPs (for example Deutsche Telekom) are accepted Certificate Authorities themselves. – Jonas Schäfer Mar 28 '17 at 6:58
  • 4
    @JonasWielicki Great point. That must be taken into consideration! – MiaoHatola Mar 28 '17 at 7:13
  • 1
    You could still have a DNS leak using a VPN which can let your ISP know which domains that you are visiting over it. – emilhem Mar 28 '17 at 8:16
  • 8
    @JonasWielicki While true, it wouldn't be for long if they used their CA status to do DPI. Between WoSign and Symantec (although to a lesser extent), I don't think they'd be spared if they generated certificates to intercept SSL connections. – Ginnungagap Mar 28 '17 at 16:15
  • 2
    @JonasWielicki: While there is a theoretical/technical threat of such abuse, it would be illegal in many jurisdictions and would be a violation of the policies CAs are required to follow in order to remain trusted by the browser. At any nontrivial scale (basically, any untargeted attacks or targeted attacks of savvy users) this would be quickly caught. – R.. Mar 29 '17 at 18:19
37

As stated by Trey, DPI can see the entire content of your network traffic. All of it. If it is plain text, then they see everything that you do.

To add on to Miao's answer:

Things DPI can see, even when you use HTTPS:

  • DNS information, e.g. https://catvideos.com/tigers - they will see https://catvideos.com
  • IP address connectivity. So, even you you HTTPS to that site with cat videos, they can see that you connected to that cat video site and downloaded 500 GB of data. They don't know what data, but they know the DNS name, and the IP address, and amount of data to that site, and to every site.
  • Ads. Many/most Ad networks do not use HTTPS, so that data is not always encrypted. This can result in a "mixed encryption" or similar warning from a browser.
  • other data: Many sites using HTTPS for login will then drop the encryption for everything else.
  • graphics: many sites won't encrypt things like their logo or various graphic or video files. They may encrypt your login and search, but not results.
  • other non-HTTPS traffic like UDP, mail, SNMP, ftp, telnet, updates to some software might not use HTTPS, etc.

With a VPN, they will still see 100% of the data. However, other than the connection to the VPN provider, they'll only see encrypted data. They will know that you downloaded 800GB from VPNco.com, but will know nothing of the data inside. Even things that are not encrypted via protocol will get encrypted since a lower level is encrypting. Now, the VPNco.com will then see your data.

With the (potential) change in US law about ISP and data privacy, combined with the (potential) loss of net neutrality, ISP's might be able to not only see 100% of your data, they could modify that data, slow or block sites they want, and might be able to sell any/all of your data to a 3rd party (as Trey states).

I'm not covering MITM (like Miao states in #1 above), since you stated ISP, I assume that you are talking about a home system and a DSL or cable modem.

https://stackoverflow.com/questions/499591/are-https-urls-encrypted

  • 10
    So ultimately it boils down to whom does a user trust​, ISP or the VPN provider? For VPNs based in, say USA when they state that they do not keep logs (like PIA) won't NSA (or CIA) force them to either keep logs or close shop? – cppanonhelp666 Mar 28 '17 at 13:14
  • 9
    @cppanonhelp666 Don't trust VPNs based in the US, because the government will do their best to extract that data. – user142755 Mar 28 '17 at 20:45
  • 2
    You can replace "(potential) change" with "change" now. – JAB Mar 30 '17 at 14:55
  • 1
    So, who do you (dis-)trust more? ISP, VPN, CIA/NSA, etc? I feel that at least with CIA/NSA, they are looking for national security issues not watching Netflix from a different region or downloading cat videos. The ISP and whomever they sell to might be interested in that data. YMMV. – MikeP Mar 30 '17 at 19:35
0

If you don't trust your ISP then your first priority should not be about packet inspection at all, but rather be to establish a trusted second channel of communication for which you can exchange information about circumventing such things.

As long as you rely solely on your ISP as the lone channel for all information exchange they can technically send you wrong login info to your VPN, even if they don't they could still take over any encryption handshakes attempted because they will always be in the middle.

They could have people in their employ who are bribed to do so or be required by law for any reason.

0

With deep packet inspection, the ISP can detect most VPN protocols (not the data encrypted in the VPN packets, just that there is VPN traffic) and block it. Some companies do this to ensure that they can decrypt all traffic (with the MITM attack and forged certificates to have DPI on SSL as well). The idea is to force you to use "insecure" communication channels by preventing everything else. Note that these "insecure" channels might be more secure , from the company's point if view, as they can do Data Leakage Prevention there.

In such a case, non-standard VPN techniques, like HTTP tunneling might be an option.

Note that the terms of use might disallow measures to circumvent DPI.

Edit: some ISPs use DPI for traffic shaping. They don't log all the transmitted data, they just check (for example) for BitTorrent traffic and assign it a lower priority or limited bandwidth. Now, they are not stealing your password, just the bandwidth you are paying for....

-1

All the above is true. One more thought: Did your ISP give you a self-signed root certificate and is it in your browser? If they did, they can open your HTTPS traffic.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.