First off, here's the full
man page entry for
ssh-keygen -o from my machine (
ssh-keygen doesn't seem to have a version flag, but the
man page is from February 17, 2016)
-o Causes ssh-keygen to save private keys using the new OpenSSH format rather than the more compatible PEM format. The new format has increased resistance to brute-force password cracking but is not supported by versions of OpenSSH prior to 6.5. Ed25519 keys always use the new private key format.
Seems pretty clear that this is just about the format of the file that's being produced. Also note that
ssh-keygen will only store Ed25519 keys in the new format, regardless of what flags you pass in.
Since both of your questions today have had the same underlying question, let's deal with it.
... these "newer formats" DSA/RSA/ECC ...
Ok, so DSA, RSA, and ECC are not different formats, they are completely different algorithms and are completely unrelated to each other.
I wish I could find a better way of explaining this, but I'm not sure I can without getting too technical. Let's try this: it's a bit like saying that
ftp are just different formats for transferring files, or
.pptx are different formats of Office documents. Calling these "just format differences" is fundamentally wrong, the software is doing a very different thing in the two cases (albeit to produce the same end-result; transferring a file, or making a pretty document).
Now let's talk about formats.
So you want to save a private key in a file? PEM is a file format for storing general cryptgraphic information, but other file formats exist. PEM can be used for many things: private keys, or certificates, or the text of an email that you want to encrypt or sign. It's just a container "cryptographic stuff".
Analogy time: saving a Word document. You could save it in the old
.doc format which is universally accepted by all versions of Office and also open source programs (PEM is also older and universally accepted), or you could use the newer
.docx format (the
-o OpenSSH format). Sometimes new features are not backwards compatible and can only be saved in the new format (like ed25519).
(Many thanks to @GordonDavisson for this analogy)
With the exception of backwards compatibility, which format you choose really has nothing to do with the contents of the file.
PEM format with an RSA key.
Note that the message starts with
-----BEGIN RSA PRIVATE KEY-----, this is standard industry-wide PEM format - any software that can read PEM will be able to read this:
$ ssh-keygen -a 100 -t rsa
$ cat .ssh/id_rsa
-----BEGIN RSA PRIVATE KEY-----
-----END RSA PRIVATE KEY-----
OpenSSH format with an ed25519 key:
In addition to it being shorter (because ECC keys just are that much shorter) notice that the message starts with
----BEGIN OPENSSH PRIVATE KEY-----, this is in OpenSSH-specific format that other software may or may not be able to read:
$ ssh-keygen -o -a 100 -t ed25519
$ cat .ssh/id_ed25519.o
-----BEGIN OPENSSH PRIVATE KEY-----
-----END OPENSSH PRIVATE KEY-----