Basically the purpose of SSL cert is for trusting each other. If I have an application that only a few clients I know can use, is generating self sign certs enough? Or do I really need to sign with a well known CA such as comodo? That is, I will generate the cert on my server, then I will distribute to my clients and they will install the cert to their browser. Would this still be susceptible to MITM? thanks
For the security of a certificate it does not matter if the certificate is self-signed, signed by some private CA or signed by a public CA. It is only important that the user trusts this specific certificate explicitly or trusts it implicitly by trusting the CA which issued the certificate.
Thus it is mostly safe if you distribute your own certificate to the users in a secure way so that these users import this certificate explicitly as trusted. It is only that this kind of system does not scale if you have lots of users and that's why we have the current system of pre-trusted root CA's inside the browsers/OS.
But, beware that when distributing your own self-signed certificate or your own CA to the client you might make the client more vulnerable against man-in-the-middle attacks. If the certificate you import as trusted can be used for signing (i.e. CA:true) then this can be used to sign certificates for any other sites, i.e. it is effectively a new trusted root CA. And because it was explicitly trusted typical security measures like certificate pinning are switched off for any certificates signed by this CA. Thus if somebody compromises your system where the private key for this certificate is stored, the attacker can mount a transparent man in the middle attacks against all users trusting this certificate.
Therefore it might be better that you don't import the certificate as trusted into the browsers certificate store. It might be better instead to connect with the browser to your site and override the warning to make an permanent exception, of course after verifying the fingerprint. With this exception the certificate is not globally trusted by the user, but the trust is limited to the specific site were you used the certificate. You still better make sure too that your certificate cannot be used as a CA, that is has not CA:true set.
The difference between self-signed and public CA signed is that third-party client browsers would receive a security warning because the self-signed certificate is not trusted: It has not been signed by a well known certificate authority, therefore the certificate could be tampered and/or the owner's identity has not been verified.
If you generate a certificate with a secure cipher and key length >2048, you would not be vulnerable to MITM. Only your identity can not be proven, but the integrity and privacy of the HTTPS communication remain.
If you have access to your clients, you can trust this self-signed certificate to avoid the warning banner by adding it to your repository of security certificates whether it is:
- Java keystore
- Apple Keychain
- Windows Certificates Snap-In
- Unix /etc/pki/ca-trust/