Should SSL be terminated at a load balancer?

Question

When hosting a cluster of web application servers it’s common to have a reverse proxy (HAProxy, Nginx, F5, etc.) in between the cluster and the public internet to load balance traffic among app servers. In order to perform deep packet inspection, SSL must be terminated at the load balancer (or earlier), but traffic between the load balancer and the app servers would be unencrypted. Wouldn't early termination of SSL leave the app servers vulnerable to packet sniffing or ARP poisoning?

Should SSL be offloaded? If so, how can it be done without compromising the integrity of the data being served? My main concern is for a web application where message layer encryption isn't an option.

Answer 1

It seems to me the question is "do you trust your own datacenter". In other words, it seems like you're trying to finely draw the line where the untrusted networks lie, and the trust begins.

In my opinion, SSL/TLS trust should terminate at the SSL offloading device since the department that manages that device often also manages the networking and infrastructure. There is a certain amount of contractual trust there. There is no point of encrypting data at a downstream server since the same people who are supporting the network usually have access to this as well. (with the possible exception in multi-tenant environments, or unique business requirements that require deeper segmentation).

A second reason SSL should terminate at the load balancer is because it offers a centralized place to correct SSL attacks such as CRIME or BEAST. If SSL is terminated at a variety of web servers, running on different OS's you're more likely to run into problems due to the additional complexity . Keep it simple, and you'll have fewer problems in the long run.

That being said

1. Yes, terminate at the load balancer and SSL offload there. Keep it simple.
2. The Citrix Netscaler load balancer (for example) can deny unsecure access to a URL. This policy logic, combined with the features of TLS should ensure your data remains confidential and tamper-free (given that I properly understand your requirement of integrity)

Answer 2

Yes, I would argue that TLS should be offloaded. I have done everything that I mention below specifically with the Citrix Netscaler, but I believe F5 should be able to do the same things.

First, you always need to make sure that you reencrypt on the other side of the load balancer, but the device decrypting TLS should be able to inspect what's going on from a security perspective. The integrity of the data should not be compromised by this approach.

Many people have said to me that reencrypting on the back end makes it just as computationally expensive, but that is not true. The expense with TLS is the building and closing of the connection, which the TLS offloader handles. On the backend you have a more persistent connection to the servers, and therefore the required resources are much lower.

Additionally, if you don't have TLS offloading then even a small DDoS attack via TLS would completely annihilate your servers. I am very familiar with this situation and TLS offloading is an incredible help from a computational perspective, and also allows you to block attacks further up the chain. For extremely large DDoS attacks, you could even split your mitigation strategy between your TLS offloader and your servers.

Answer 3

To inspect the data which goes within a SSL connection, then either of these must be true:

• The tunnel ends on the machine which does the inspection, e.g. your "load balancer".
• The inspection system knows a copy of the server's private key, and the SSL connection does not use ephemeral Diffie-Hellman (i.e. the server does not allow the cipher suites which contain "DHE" in their name).

If you follow the first option, then data will travel unencrypted between the inspection system (the load balancer) and the clusters, unless you reencrypt it with some other SSL tunnel: main SSL connection is between client browser and the load balancer, and the load balancer maintains a SSL link (or some other encryption technology, e.g. a VPN with IPsec) between itself and each of the cluster nodes.

The second option is somewhat lighter, since the packet inspector just decrypts the data but does not have to reencrypt it. However, this implies that all cluster nodes are able to do the full SSL with the client, i.e. know a copy of the server private key. Also, not supporting DHE means that you will not get the nifty feature of Perfect Forward Secrecy (this is not fatal, but PFS looks real good in security audits so it is a fine thing to have).

Either way, the node which performs deep packet inspection must have some privilege access into the SSL tunnel, which makes it rather critical for security.