0

I have 3 VPS based on Ubuntu 18.04 server OS and would like to open particular ports on each servers. The ports, that I would like to open are: enter image description here

Then let's take a look on the environment: enter image description here

As you can see on the image, every server has a public IP address.

Now, I would like configure the UFW firewall, so that only those 3 IP addresses are allowed to communicate through the ports, that are listed above. For example, the server with the IP 120.88.41.175 has the permission to access the server with the IP 120.88.41.176 through the ports, that I've listed above. The access should be bidirectional.

But the server with the IP address 120.88.42.175 has no permission to access the server with IP address 120.88.41.176.

enter image description here

I know, that UFW offers settings to allow particular IP to access particular port on the server, for example:

sudo ufw allow from 120.88.41.175 to any port 6443* proto tcp

Now the server 120.88.41.175 has access to the server 120.88.41.176 through port 6443*.

Then a bad person comes and try to access the server 120.88.41.176 through the port 64435 and he/she identifies themself as 120.88.41.176. Now he/she can do bad stuff on the server.

The question is, how can I protect my VPS from such as violation but at the same time, keep the open the ports? I need to open the ports because I am going to build a kubernetes cluster.

  • Aren't you basically asking how to prevent or detect IP spoofing within TCP connections? First, spoofing the source IP in TCP is very hard although not completely impossible. Seconds, your 120.88.41/24 network should be protected by a router/firewall which prevents such addresses to show up as source on the internet facing interface since they are not supposed to be there. – Steffen Ullrich Jan 29 at 17:51
2

So, to start, let's look at the architecture. Many VPS providers aren't good about this, but if you can communicate between your VPS' on an internal network using private IPs, that's preferable. Digital Ocean will let you do private networking and accomplish a lot of the inter-connectivity without traversing the public Internet. If you can do this, you'll remove a ton of attack surface by limiting potential MitM attacks and making IP spoofing untenable (or at least less-tenable) from the Internet.

Next you would want to avoid having to expose the API server to the public Internet (or at least I would). The kubernetes API server is fairly robust and uses reasonable authentication, but it presents a large attack surface and potentially large damage if it's compromised. So I'd recommend having a Bastion Host or VPN server that allows you to connect into the kubernetes environment where you can administer your cluster. Just don't store any authentication mechanisms on this bastion hosts, make it a dumb jump-box in case it's compromised.

To avoid exposing all of the nodeport services, you can use a Load Balancer, if your VPS provider has one, probably you should use that, otherwise a dedicated nginx box would be best. However, for tighter integration, you might expose 443 on each of your worker nodes and use the nginx-ingress controller on the cluster itself, this way you can use a lot of the built-in integrations, but still control what services are externally exposed.

Once you have those two things taken care of, you could probably get away with just allowing all traffic between your kubernetes nodes within the cluster. But it would be better to batten the hatches as it were.

Let's define the servers/VPS into the following groups

  • Etcd nodes
  • API Servers
  • Control plane (all of the Etc nodes, schedulers, controllers, api servers)
  • Worker nodes
  • Bastion Host
  • Load Balancer

you would want to implement the following rules using UFW and (if it exists) your VPS' security groups/cloud firewalls/etc. These rules assume a stateful firewall like UFW or AWS Security Groups. If you're using AWS Security groups, you can have named security groups for each of the above groups and have them reference each other and not specify IP addresses at all. Using UFW, you will have to list IP addresses.

  • Control Plane and Bastion Host -> API servers over TCP/6443
  • Etcd and API Servers -> Etcd Nodes over TCP/2379-2380
  • Load Balancer and Worker nodes and Control Plane -> Worker nodes over TCP/30000-32767
  • Control Plane -> Control Plane and Worker nodes over TCP/10250
  • Control Plane -> Control Plane over TCP/10251-10252

This could be broken down further, but I think this is a reasonable configuration. I don't think the Control Plane needs access to the nodeport nodes, I don't think the etcd cluster needs access to the kubelet, and so on. Proceed down the rabbit hole as far as you like. To lighten up the security posture you can expose the nodeports and the API server to the world, just make sure you understand the ramifications of doing so.

Attacks against this architecture

As you mention, if your cluster is talking over the public Internet, anyone can identify themself as a whitelisted IP address. Suppose an attacker sends a request to Etcd at 120.88.41.177 and says that their IP is 120.88.41.176, but their real IP is 120.88.41.175. The request will bypass UFW just fine, but the server will also send the reply to 120.88.41.176 and not to the attacker. Unless the attacker can intercept the traffic, the attacker never can see any of the replies.

Also, for this to work, the IP addresses of your servers would have to be known to the attacker (which may or may not be a tall order).

Next, the cluster-level communications (Between the Kubelets, API servers, Etcd cluster, controllers, and schedulers) all run over HTTPS by default, using client-side TLS authentication. (EDIT It looks like Etcd with authentication is not support by default as of k8s 1.17, so limiting network access to Etcd from ONLY the API server(s) is very important). So in order for the attacker to communicate with anything within your cluster, they also have to compromise your CA or a client key somewhere in your infrastructure. At which point, if an attacker compromises a client key, they probably have some access to one of your nodes and the network security is moot.

This is why I would suggest some variation of the above suggestions for your network, and then focus your efforts on good RBAC controls to limit what actions each pod can do, and limit what actions each node can do. Or add some network policies within kubernetes to limit the access of the pods. At the host/VPS level, the networking is pretty straightforward, the API server has to be reachable by all of your nodes, so you can't really protect against a compromised node/pod using UFW.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.