Are OpenSSH vs OpenSSL public key format names mismatched?

Question

I generated a key pair vis OpenSSH and later wanted to generate PEM encoded public key. I noticed that generated public key differs when generated by OpenSSH vs OpenSSL when using same format name (PEM)... After some testing (see below), I found that public keys are the same only when generated a) as PKCS8 in OpenSSH and b) as PEM in OpenSSL

1. Why is PEM output of OpenSSH different than PEM format of OpenSSL?
2. If this is just a parameter naming mismatch, am I correct in my assumption that to get the same public key, I have to use PCKS8 in OpenSSH and PEM in OpenSSL?

Here is my test:

# -----------------------------------------------------------------------------
# Using OpenSSH
# -----------------------------------------------------------------------------

# generate test key pair 
[root@localhost key-test]# ssh-keygen -t rsa -b 2048 -f test.ssh

#
# get MD5 fingerprint of generated public key 
#
[root@localhost key-test]# cat test.ssh.pub | openssl md5 -c        
(stdin)= 3e:b6:6f:95:d2:1d:ca:7a:f3:88:2e:3a:ca:c3:b7:a3

#
# get MD5 fingerprint of generated public keys (RFC4716, PEM, and PKCS8 formats)  
#

# header => ---- BEGIN SSH2 PUBLIC KEY ----
[root@localhost key-test]# ssh-keygen -y -e -m RFC4716 -f test | openssl md5 -c
(stdin)= 0d:da:5b:27:e9:cd:32:25:e5:c8:4e:2b:c2:0f:47:aa

# header => -----BEGIN RSA PUBLIC KEY-----
[root@localhost key-test]# ssh-keygen -y -e -m PEM -f test | openssl md5 -c       
(stdin)= c5:38:c0:41:f8:76:41:7b:49:03:42:c6:21:dd:1c:2f

# header => -----BEGIN PUBLIC KEY-----
[root@localhost key-test]# ssh-keygen -y -e -m PKCS8 -f test | openssl md5 -c  
(stdin)= 84:e8:54:5f:16:9f:92:48:67:99:54:65:13:ad:64:ad


# -----------------------------------------------------------------------------
# Using OpenSSL
# -----------------------------------------------------------------------------

#
# get MD5 fingerprint of the generated public key (PEM format)
#
# header => -----BEGIN PUBLIC KEY-----
[root@localhost key-test]# openssl rsa -in test -pubout -outform PEM | openssl md5 -c            
(stdin)= 84:e8:54:5f:16:9f:92:48:67:99:54:65:13:ad:64:ad

Answer 1

A little step:

$ ssh-keygen -y -e -m PKCS8 -f /home/wolf/.ssh/id_rsa | openssl asn1parse
    0:d=0  hl=4 l= 290 cons: SEQUENCE          
    4:d=1  hl=2 l=  13 cons: SEQUENCE          
    6:d=2  hl=2 l=   9 prim: OBJECT            :rsaEncryption
   17:d=2  hl=2 l=   0 prim: NULL              
   19:d=1  hl=4 l= 271 prim: BIT STRING        

$ openssl rsa -in /home/wolf/.ssh/id_rsa -pubout -outform PEM | openssl asn1parse
writing RSA key
    0:d=0  hl=4 l= 290 cons: SEQUENCE          
    4:d=1  hl=2 l=  13 cons: SEQUENCE          
    6:d=2  hl=2 l=   9 prim: OBJECT            :rsaEncryption
   17:d=2  hl=2 l=   0 prim: NULL              
   19:d=1  hl=4 l= 271 prim: BIT STRING        

$ ssh-keygen -y -e -m PEM -f /home/wolf/.ssh/id_rsa | openssl asn1parse
    0:d=0  hl=4 l= 266 cons: SEQUENCE          
    4:d=1  hl=4 l= 257 prim: INTEGER           :DEE24467850E0F877429474448807D2000B30E889BE6CA3BA8561B69D925F1C0AFA9A536A7EE95D077D5E5D6D3BA90C4C5E3B547B6872AD3F795816A54A7170EEB3454351424FD751EFA9A07F277C9F4987FCC94F9C06658D95567DD2A9E99AAF130BD86481AD34DA84DB1E91F1954043137E53816BA9AF49962E0BAC4B51FECBA45F4C20FC66F6AAAB910C576FD9AED5AE03BD77A2B9D51662DCF1725FEAD7263F29F170E8953FA3CE9C3E4B7A062F840C3273104946C266DC06AFAB2EF03DD8ABFCA0EF0B25AD9D07CFC9DADB35284FF6C0C9B73AD8E744B575AA833737DA8B1415615811E5FE95E6FA144B91A05B240DF20A29D12FD29AA9FF0A2E90F4973
  265:d=1  hl=2 l=   3 prim: INTEGER           :010001

$ openssl rsa -in /home/wolf/.ssh/id_rsa -RSAPublicKey_out -outform PEM | openssl asn1parse
writing RSA key
    0:d=0  hl=4 l= 266 cons: SEQUENCE          
    4:d=1  hl=4 l= 257 prim: INTEGER           :DEE24467850E0F877429474448807D2000B30E889BE6CA3BA8561B69D925F1C0AFA9A536A7EE95D077D5E5D6D3BA90C4C5E3B547B6872AD3F795816A54A7170EEB3454351424FD751EFA9A07F277C9F4987FCC94F9C06658D95567DD2A9E99AAF130BD86481AD34DA84DB1E91F1954043137E53816BA9AF49962E0BAC4B51FECBA45F4C20FC66F6AAAB910C576FD9AED5AE03BD77A2B9D51662DCF1725FEAD7263F29F170E8953FA3CE9C3E4B7A062F840C3273104946C266DC06AFAB2EF03DD8ABFCA0EF0B25AD9D07CFC9DADB35284FF6C0C9B73AD8E744B575AA833737DA8B1415615811E5FE95E6FA144B91A05B240DF20A29D12FD29AA9FF0A2E90F4973
  265:d=1  hl=2 l=   3 prim: INTEGER           :010001

$ ssh-keygen -y -e -m PKCS8 -f /home/wolf/.ssh/id_rsa | tail -n +2 | head -n -1 | base64 -d | openssl asn1parse -inform DER -i -strparse 19
    0:d=0  hl=4 l= 266 cons:  SEQUENCE
    4:d=1  hl=4 l= 257 prim:  INTEGER           :DEE24467850E0F877429474448807D2000B30E889BE6CA3BA8561B69D925F1C0AFA9A536A7EE95D077D5E5D6D3BA90C4C5E3B547B6872AD3F795816A54A7170EEB3454351424FD751EFA9A07F277C9F4987FCC94F9C06658D95567DD2A9E99AAF130BD86481AD34DA84DB1E91F1954043137E53816BA9AF49962E0BAC4B51FECBA45F4C20FC66F6AAAB910C576FD9AED5AE03BD77A2B9D51662DCF1725FEAD7263F29F170E8953FA3CE9C3E4B7A062F840C3273104946C266DC06AFAB2EF03DD8ABFCA0EF0B25AD9D07CFC9DADB35284FF6C0C9B73AD8E744B575AA833737DA8B1415615811E5FE95E6FA144B91A05B240DF20A29D12FD29AA9FF0A2E90F4973
  265:d=1  hl=2 l=   3 prim:  INTEGER           :010001

The openssl man page calls one the PEM format and the other the RSAPublicKey format.

What ssh-keygen calls PKCS8 and openssl rsa calls PEM is a wrapper around what ssh-keygen calls PEM, that adds an ASN.1 OID for "rsaEncryption". The last example parses the DER bitstring that starts at offset 19 to convince ourselves that it's the same exact data.

The format is described in PKCS#1.

PKCS#8 is here, and talks about storing private keys, not public keys.

I can't help answering what is the more correct name for each of the two formats, though I would call them by the ASCII string in the header.

I like the format that encodes the fact that it's an RSA public key inside of it, and so does not require externally encoding it (like the "ssh-rsa" prefix used in ~/.ssh/authorized_keys), though it is a little bigger. Using a self describing format, a program can learn to read ECDSA and EdDSA keys without inventing a new way to specify that the data is a public key but not an RSA public key.