I don't have much experience in computer security and I am developing a server for a project. I'm using a tcp connection and UDP packets for some real time data transmission. I want both of these channels encrypted using a self signed cert. What I have planned so far is.

  1. Multiple servers are started for redundancy.
  2. Each server generate a self signed cert when starting up and registers the public key in a service discovery agent (ETCD maybe).
  3. There is an API( which is https, using a cert signed by a valid CA). Lets call this management API
  4. Clients first query management API, which responds with an IP:Port and the public key of the desired server to connect.
  5. Clients connect(TCP TLS) to the server IP with the public key as CA (which validates the TLS connection).
  6. The server then generates a new AES GCM session, generates a 256 bit key and sends the key through the TCP TLS connection.
  7. Each UDP packet to and from the server is encrypted using this key (using AES GCM)
  8. Client uses the initialized tcp connection and UDP channel to communicate with the server.


  1. Is this approach secure ?
  2. What are security vulnerabilities of this plan ?
  3. If there are vulnerabilities, how can I fix them?
  4. is AES GCM mode is a good fit for UDP packet encryption?

Notes :

  1. Server is written using java (Netty), client is cpp(boost asio).
  2. The UDP packets contain time sensitive data, therefore low latency is a priority.
  3. I have looked into two encryption algorithms (AES, Salsa). planning to use AES for UDP encryption because running "gnutls-cli" benchmarks in EC2 reveals AES is way faster on modern CPUs (source).
  4. UDP data is VOIP data.
  • 1
    You're using a home-grown protocol for the UDP part. Don't. Use DTLS or use UDP inside a WireGuard VPN connection (which I would consider better than DTLS and preferable if you can make it work). – Z.T. Mar 29 at 22:00
  • 1
    While it is true that AES-128-GCM is faster than CHACHA20-POLY1305 on hardware that has AESNI and CLMUL (meaning all x86 hardware from the last decade), ChaPoly is fast enough. It's a gigabyte per second per core on my laptop, vs almost 5 gigabytes per second per core for AES-128-GCM. – Z.T. Mar 29 at 22:04
  • 1
    From a deployment perspective, usually you want clients to only ever know about one certificate (chain), just to make things easier. Is there a reason you're UDP-ing directly to your "real" servers? As opposed to setting up an API gateway or reverse proxy, where your clients only ever know about one public-facing endpoint, and communications between the API gateway and backend servers are encrypted with internal keys. – Clockwork-Muse Mar 30 at 0:06
  • @Z.T. thanks for quick reply. I will look into DTLS. I didn't want to in the first place since I could not find solid DTLS implementation for Java. – Bhanuka Yd Mar 30 at 11:31
  • @Clockwork-Muse I connect directly to the server since I use UDP for some time sensitive data. AFAIK API gateways are HTTP and support WebSockets but not UDP. – Bhanuka Yd Mar 30 at 11:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.