The distinction depends on some details about the self-signed certificate and how the browser and the OS will react to it.
A normal certificate includes a
Basic Constraints extension which contains a flag telling whether the certificate is for a CA or not; when the flag is
TRUE then the certificate is considered acceptable as an issuer for other certificates. If the extension is altogether missing, then the certificate is to be considered as non-CA.
Root CA certificates are "special"; they are not true certificates. They are trust anchors; a TA is, nominally, a public key coupled with a name. It is traditional to encode trust anchors as certificates, and since a certificate includes a field for a signature, some bytes must be stored there; there again, Tradition says that we shall make the certificate self-signed. Nobody really checks that signature, though; junk bytes of approximately the right size would serve just as well.
The tricky point is then that the interpretation of the certificate extensions, if any, found in such a "root CA certificate", is non-standard. In particular, some historical root CA date from before the invention of the
Basic Constraints extension; as true certificates, they would be interpreted as "not CA", but since they are trust anchors disguised as certificates, the normal interpretation rules do not apply.
So on some browsers or OS, a non-CA certificate (lacking a
Basic Constraints extension, or even featuring one with the
cA flag set to
FALSE), once installed in the "root CA" store, may very well begin to be accepted as a CA. Thus, confidentiality of the private key for any certificate installed in the user's trust store is rather critical.
What's the difference in your case ? After all, in your situation, there are two choices:
- You create a self-signed certificate for your server, and the user installs that certificate as "trusted".
- You create a self-signed CA certificate, that the user installs as "trusted". You issue a normal certificate for your server with this CA.
In both cases, the user surrenders his security to some certificate from you, that the user installs as "trusted". So what would it change ?
The difference is that if you use a custom CA, you can store the corresponding private key with good protection. For instance, you do the CA business on a laptop which is kept offline at all times. There is no possibility of a remote exploit when there is no network at all. Root CA should always be offline, and used only as part of manual procedures and USB keys for data transfer. On the other hand, your server is, well, a server, so it is online and intrinsically exposed. An attacker who hacks into your server may get a copy of the server's private key. If the corresponding public key has been installed as "trusted root" in the users' machines, then the attacker gains a lot of leverage on these machines as well.
Briefly said, making a custom CA distinct from the server's certificate (as used all day long) allows for better protection of the private key, and thus a lower risk of catastrophic attacker's power escalation.
There are browsers which are not as vulnerable. What you really want, with a self-signed server certificate, is that the users trust that certificate only for SSL connections (not as a CA), and only for that specific server. Firefox knows how to do that; it is called a "security exception" in Firefox terminology (see the Preferences -> Advanced -> Certificates -> View Certificates, then the "Servers" tab).