Let me provide a high level picture. Clearly, you understand the need for secure communications, be it SSH or HTTPS. Secure communications mean that the channel is encrypted.
Generally speaking all encryption algorithms fall into one of two categories:
- Symmetric encryption. One key. Same key is used to encrypt and decrypt. Fast. E.g. AES.
- Asymmetric encryption. Two keys. Either may be used to encrypt, but only the other one can decrypt. Much slower than symmetric algorithms. E.g. RSA.
Symmetric encryption is fast and therefore suited for communications involving a lot of data between two parties. It uses the same key for encryption and decryption - this key is analogous to your concept of a very long password. Problem: how do you share your key/password in the first place? Turns out you cannot use a secure channel that is built solely on a symmetric encryption algorithm without figuring out a way to first share your key/password.
This is where asymmetric algorithms come in, but are considerably slower than symmetric algorithms. Impractical to transmit large amounts of data, but fine if you transmit or exchange something small like a symmetric encryption key/password. Once that's done you can now use symmetric encryption for communications.
One of the two keys for asymmetric encryption is designated the public key and the other the private key. The public key may be distributed to everyone, but private key must be kept secret.
You (has pub) Server (has prv + pub)
asym-encrypt(pub, sym-key/pwd) ----> asym-decrypt(prv, encrypted-data) => sym-key/pwd
pub = public key, prv = private key
In any case this explanation is a simplification of what SSH actually does. Two other things worth highlighting:
Diffie Hellman is the typical key exchange asymmetric algorithm. With Diffie Hellman you don't actually need to create a symmetric key. Both parties create the symmetric key together during the key exchange, which is a nice security feature. See "Diffie-Hellman Key Exchange" in plain English.
In my explanation I assumed that you found the server's public key, and that you trust it is the correct one. But you really should be careful about which public keys you trust. To trust a public key, it should be digitally signed. A signed public key is also known as a certificate.
Hopefully this clears up the questions about long password and public keys, and has enough information for you to dig deeper in a more meaningful fashion.