Why do systems that do password authentication actually send the
password over the wire? Why not just have the server issue a
Let me reverse the question. What would you gain by not sending the password over the wire? The answer will often be nothing.
The primary place where we use password is on the internet. When you visit a web page, this page is controlled by the server and hence you can consider it an extension of the server. If the server is compromised or evil, as soon as you type the password on the page, you can consider it compromised and this password should be changed.
In your question you seem to worry about MitM attack, but only consider the MitM that can append when you submit the form and not when you receive the page, but both are as deadly. It doesn't matter if your password is not "supposed" to be sent or not, as soon as it's typed it's same thing.
So is it totally useless to use challenge-response authentication on a webpage?
If you use TLS, I would say yes. Remember the client is just an extension of the server.
If you don't use TLS, it can help against passive eavesdropping but is still useless against active MitM; where you can modify the content of the page. The fact that active MitM is much harder to pull off than passive eavesdropping was a reason that motivated the creation of such scheme. See digest authentication.
Why is SSH different?
Because the server doesn't control the client. The client is completely separated from the server. This means that when I typed my secret password in the client, even if I response to a challenge from an evil server, my secret password is still safe.
It's the same thing with your credit card. The smart chip on your credit card contains a secret key that is used to answer a challenge-response every time you make a purchase. This secret key is protected even if you make a purchase from an evil terminal.
Note : An advantage of the challenge-response mechanism is also that it lets the user reuse the same password with multiple server. This point is need for credit card for example as they will use many terminals (servers) and only have one password.
Challenge-response mechanism only work when the server doesn't control the client.
This means that if we wanted to use challenge-response authentication for webpage, it would need to be implemented at the browser level and not the web page level. Whether that would be a good idea or not is a completely different question.
I believe this is because then you can't salt and hash the password on
the server side.
It's true that if you use symmetric encryption, you won't be able to protect the password correctly on the server as you need it to verify the answer of the challenge. That's why you use asymmetric encryption for challenge-response scheme. For example, both SSH and your credit card are using asymmetric encryption. With asymmetric encryption, you can store the public key on the server and the user can keep it's private key... private.