For a DSA key pair, use this:
openssl dsaparam -genkey 1024 -out dsakey.pem
where "1024" is the size in bits. The first DSA standard mandated that the size had to be a multiple of 64, in the 512..1024 range. Then another version deprecated sizes below 1024, so a valid DSA key size had length 1024 bits and nothing else. Current version specifies that a valid DSA key has length 1024, 2048 or 3072 bits. OpenSSL accepts other lengths. If you want to maximize interoperability, use 1024 bits.
For an ECDSA key pair, use this:
openssl ecparam -genkey -out eckey.pem -name prime256v1
To see what curve names are supported by OpenSSL, use:
openssl ecparam -list_curves
(For optimal interoperability, stick to NIST curve P-256, that OpenSSL knows under the name "prime256v1".)
Once you have a DSA or ECDSA key pair, you can generate a self-signed certificate containing the public key, and signed with the private key:
openssl req -x509 -new -key dsakey.pem -out cert.pem
(Replace "dsakey.pem" with "eckey.pem" to use the EC key generated above.)
For Diffie-Hellman (with or without elliptic curves), things are more complex, because DH is not a signature algorithm:
- You will not be able to produce a self-signed certificate with a DH key.
- You cannot either make a PKCS#10 request for a certificate with a DH key, because a PKCS#10 request is supposed to be self-signed (this self-signature is used as a proof of possession).
While OpenSSL, the library, has the support needed for issuing a certificate which contains a DH public key; this page may contain pointers. The challenge is to convince OpenSSL, the command-line tool, to do it. In the jungle of the OpenSSL documentation, I have not found a complete way to do it. Key pairs are easy enough to generate, though.
To generate a DH key pair, with the OpenSSL command-line tool, you have to do it in two steps:
openssl dhparam -out dhparam.pem 1024
openssl genpkey -paramfile dhparam.pem -out dhkey.pem
For an ECDH key pair, use this:
openssl ecparam -out ecparam.pem -name prime256v1
openssl genpkey -paramfile ecparam.pem -out ecdhkey.pem
However, it so happens that the format for certificates containing ECDH public keys is completely identical to the format for certificates containing ECDSA public keys; indeed, the format contains "an EC public key" without indication of the intended algorithm (ECDH or ECDSA). Therefore, any private key and certificates for ECDSA (private key for generating ECDSA signatures, certificate self-signed or signed by any other CA) will be fit for ECDH-* cipher suites.
The one case that I don't know how to produce with the OpenSSL command-line tool is a static Diffie-Hellman (non-EC) certificate. Note, though, that OpenSSL does not support SSL/TLS with static DH cipher suites either, so even if you could produce the certificate, it would not work with OpenSSL.
(And, in fact, nobody uses static DH in practice.)