How should certificates be shared between a proxy and a frontend?

Question

At work, I've been asked to look into creating a reference topology for a WebSocket chat application, which includes a proxy between the web-facing frontend server and the backend. At the moment, I am still running the frontend and backend on the same server for development, but I've temporarily set the WebSockets to point to the proxy, which is on a completely separate server in the lab. Both servers are on an internal network, and each has a separate, self-signed certificate.

When I was testing the WebSocket over TLS, it failed due to the cert being self-signed (without showing any dialogue to make an exception) until I connected directly to the proxy and made an exception for the certificate. After that, it worked perfectly fine, but in practice the visitor probably wouldn't be allowed to do that. So, I need to find out how to certify both at the same time.

As far as I can tell, the main ways of doing this are:

• Put the proxy and frontend on different subdomains (e.g. frontend.domain.com and proxy.domain.com), and use a wildcard certificate for the site. This would be quicker and easier to work with in the lab, but the IETF discourages this.
• Use a single cert with Subject Alternative Names, and include both frontend.mydomain.com and proxy.mydomain.com under the SAN field for www.mydomain.com. As far as I can tell, this shares the problem of vouching for rogue/buggy domains.
• Use entirely separate certs for the two servers, like I'm already doing. It's more hassle to work with, especially with self-signed certs, but if one domain is compromised, that's only one point of failure. My main question here is that if the proxy and frontend certs weren't self-signed, would this be more likely to work?

Is there anything I've missed with the above, or are there other methods?

EDIT 22/07/2015: As an aside, while sending an XMLHttpRequest via the same mechanism, I found I had to turn off the cert verification between the proxy and the backend to allow the SSL handshake to go ahead, so I'm wondering if my self-signed certs were the real cause of the problem. However, that probably doesn't solve my issue above.

Answer 1

If you have a self-signed certificate you need your clients to trust it.

Alternatively you can have the client not check whether the certificate is trusted, but that's a terrible idea.

How you do that depends on your clients. If you have something like this:

[user-agent]---------[frontend]---------[proxy]---------[backend]

The user-agent (probably a browser) is the client for frontend, frontend is the client for proxy and proxy is the client for backend.

Each client here needs to trust the certificate offered by the server, either because it's signed by an already trusted entity, or because you add the self signed cert to the truststore.

So yes, self-signed can give you trouble

If the proxy is there for scalability reasons there will be more backend hosts in the future.

If all communications behind the frontend are internal to your organization, you can setup your own internal CA and trust its root certificate in the frontend and proxy.

This way you can add new hosts internally each with their own certificate and the clients will automatically trust it.

Using different certificates for different servers is usually a good idea, independently of whether you deploy your own CA or not. When a key is compromised you need to revoke the certificates for that key, and any host that shared certificates with the compromised one will require new certificates as well.