From the point of view of the service: No, a passphrase protected SSH private key is not multifactor authentication.

The SSH server has no way to know whether the private key is encrypted or not, and has no way to know what that current passphrase may be in any case. The closest that the server can get is, if the key pair is generated on the server, it can capture the passphrase at that time. (This would be very unusual, and I'd question the security of any system that does this.) Once the private key has left the server, though, the only thing it can assert is that, at one point someone used the passphrase to decrypt the key. The server does not know if it was decrypted seconds ago as part of authenticating or if they private key is currently sitting on the client machine's disk completely unencrypted.

So, while it is a good practice to encrypt the private key with a passphrase, the authentication handshake between the client and server do not use that passphrase, thus the passphrase is not part of authentication.

As to whether or not the private key is something you have or something you know, I argue that it is something you have, because you are not passing the private key directly to the server, you are proving that you have the private key:

The authentication handshake goes like this:

1. The client selects a key to use and sends the key's ID to the server.
2. The server gets the public key from ~/.ssh/authorized_keys, generates a nonce, and encrypts it with that public key.
3. The client decrypts the nonce with its private key, then MD5 hashes it with the shared session as salt.
4. If the server gets the expected hash back, the user is authenticated.

This is a different process than passing a password; you are proving more than just knowledge, you are proving that you have a system capable of performing decryption on a message encrypted with a specific public key.

In physical security, something you know would be implemented with a challenge-response: The guard calls out a word, and you respond. (This also authenticates the guard. Don't give the password of the day to someone just because they're wearing a uniform.)

Similarly in physical security, something you have is a key. Yes, the key contains information that is easy to copy and could even be memorized, but unless that data is cut into a physical object, the data does no good. With a key, you are proving more than just knowledge, you are proving that you have an object capable of lifting the tumbler's pins to the correct height (whether that's the intended key, a copy, or a set of lock picks doesn't really matter unless you also have a guard). Just as a private key could be memorized by a very determined person, it's useless to just spam the private key's bits across the wire like you would a password.

From the point of view of the service: No, a passphrase protected SSH private key is not multifactor authentication.

The SSH server has no way to know whether the private key is encrypted or not, and has no way to know what that current passphrase may be in any case. The closest that the server can get is, if the key pair is generated on the server, it can capture the passphrase at that time. (This would be very unusual, and I'd question the security of any system that does this.) Once the private key has left the server, though, the only thing it can assert is that, at one point someone used the passphrase to decrypt the key. The server does not know if it was decrypted seconds ago as part of authenticating or if they private key is currently sitting on the client machine's disk completely unencrypted.

So, while it is a good practice to encrypt the private key with a passphrase, the authentication handshake between the client and server do not use that passphrase, thus the passphrase is not part of authentication.

As to whether or not the private key is something you have or something you know, I argue that it is something you have, because you are not passing the private key directly to the server, you are proving that you have the private key:

The authentication handshake goes like this:

1. The client selects a key to use and sends the key's ID to the server.
2. The server gets the public key from ~/.ssh/authorized_keys, generates a nonce, and encrypts it with that public key.
3. The client decrypts the nonce with its private key, then MD5 hashes it with the shared session as salt.
4. If the server gets the expected hash back, the user is authenticated.

This is a different process than passing a password; you are proving more than just knowledge, you are proving that you have a system capable of performing decryption on a message encrypted with a specific public key.

In physical security, something you know would be implemented with a challenge-response: The guard calls out a word, and you respond. (This also authenticates the guard. Don't give the password of the day to someone just because they're wearing a uniform.)

Similarly in physical security, something you have is a key. Yes, the key contains information that is easy to copy and could even be memorized, but unless that data is cut into a physical object, the data does no good. With a key, you are proving more than just knowledge, you are proving that you have an object capable of lifting the tumbler's pins to the correct height. And just as the passphrase on a private key is not part of the authentication, whether the tool used to turn the tumbler is the intended key, a copy, or a set of lock picks is also not part of the authentication.