We have a Kubernetes deployment with an application that need to be on a VPN. We implement this requirement by running openvpn-client in a sidecar container within the pod with elevated capabilities:
securityContext: capabilities: add: - NET_ADMIN
We'd like to better understand the impact of this, and how exposed we'd be if this container were compromised. We want to be confident that code exec in this container couldn't view or modify packets or network configuration in other pods, or on the host node.
My current hypothesis is that since each pod has an isolated network namespace, giving
CAP_NET_ADMIN to a container in the pod just provides the capability within that namespace.
However, I haven't been able to find any documentation that's definitively discusses the impact of using
securityContext to assign capabilities to containers. There's a few pieces of documentation - outlined below - that strongly imply that Kubernete / Docker will provide sufficient isolation here, but I'm not 100% certain.
The pods documentation on resource sharing  gives a hint here:
The applications in a Pod all use the same network namespace (same IP and port space), and can thus “find” each other and communicate using
localhost. Because of this, applications in a Pod must coordinate their usage of ports. Each Pod has an IP address in a flat shared networking space that has full communication with other physical computers and Pods across the network.
The networking documentation on the network model has this to say:
Kubernetes IP addresses exist at the
Podscope - containers within a
Podshare their network namespaces - including their IP address. This means that containers within a
Podcan all reach each other’s ports on
localhost. This also means that containers within a
Podmust coordinate port usage, but this is no different from processes in a VM. This is called the “IP-per-pod” model.
Finally, I note that
pod.spec.hostNetwork is configurable, and defaults to false:
$ kubectl explain pod.spec.hostNetwork KIND: Pod VERSION: v1 FIELD: hostNetwork <boolean> DESCRIPTION: Host networking requested for this pod. Use the host's network namespace. If this option is set, the ports that will be used must be specified. Default to false.