HTTPS is HTTP-within-SSL. SSL is a tunnel protocol: it works over an existing bidirectional stream for data, and provides a bidirectional stream for data. The two parties involves in SSL are the client and the server, which are two roles within the SSL protocol; it is not required that these roles map to the notions of "client" and "server" of the underlying transport protocol.
For instance, a setup can be imagined, in which the client system (C) initiates a TCP connection to the server (S), and then the server initiates a SSL handshake, acting as the SSL client (i.e. sending the
ClientHello message, instead of waiting for an incoming
ClientHello). This reverses the roles of both machines, and also the security guarantees: the machine S will have a good idea of the identity of the connected client C, but the client C will not be sure of what server S it is talking to (an attacker could have intercepted and redirected the communication). Depending on the context, this may or may not be appropriate.
However, this departs from HTTPS, in which the TCP client is also the SSL client, and that client expects the server to show a certificate, which the client will validate against its known, trusted CA, and which contains the expected server name (as extracted from the URL, see section 3.1). Correspondingly, existing clients (Web browser) do not support reversal of SSL roles. If your situation calls for using browsers, then you must, of course, use only the functionality available in browsers.
SSL does support a few certificate-less cipher suites. The "DH_anon" cipher suites are deemed weak, because they imply no authentication at all (thus, Man-in-the-Middle attacks are possible). The PSK cipher suites imply mutual authentication of both client and server with regards to a shared secret. When the shared secret is of low entropy (say, it is a password), SRP cipher suites are better.
There again, these cipher suites are not (yet) available in mainstream browsers (although some people are working on it). They require a shared secret (key or password), a condition which may or may not be easy to achieve in your specific context.
If knowledge of the server identity is unimportant, then you can give the server a self-signed certificate, along with instructions for clients on how to make their browser accept the server certificate without cringing too loudly (see this question as a starting point). This will map to "normal SSL", which has two benefits:
- Existing browsers support that.
- When the server presents a certificate, however bogus, it is then allowed to ask, in return, for a client certificate, yielding the kind of authentication that you are looking for. And Web browsers do support client certificates for SSL.
Note that the self-signed certificate contains the server public key. Though this public key won't be validated, it will still be used to power the key exchange, so you must use an appropriate key type and length (say, RSA 2048). Alternatively, use one of the "DHE" cipher suites, in which case the server public key is used only for signatures, not to actually protect the data, so (in your specific case), its size and secrecy becomes unimportant.