The ssh host key warning means that the public key that the server is presenting now is not the same as it was when you connected to this server previously. This means that you could be connecting to a server being run by an attacker - either in an attempt to spoof the 'real server', or in an attempt to MITM the connection to the real server. If you ignore the warning, then you enable the attacker to proceed with the attack.

One benefit to public key authentication over password authentication in this case is that if the attack succeeds, the attacker does not come away with anything useful for authenticating with the real server. With public key authentication, the client only divulges his public key (which is public anyway), instead of divulging his password.

The ssh host key warning means that the public key that your ssh client now sees for the server is not the same as the public key that your client saw for this server previously. This means that you could be connecting to a malicious server being run by an attacker - either in an attempt to spoof the 'real server', or in an attempt to MITM the connection to the real server. If that's the case, and you ignore the warning, then you will be enabling the attacker to proceed with the attack.

One benefit to public key authentication over password authentication in this case is that if the attack succeeds, the attacker does not come away with anything useful for authenticating with the real server. With public key authentication, the client only divulges his public key (which is public anyway), instead of divulging his password.