Alice's public key is useless to Bob unless he can prove that the public key came from Alice. To do this, Bob and Alice can establish a secure channel to exchange Alice's the public key, but this same channel can equally be used to exchange a symmetric pre-shared key, so this is effectively identical.
A 'secure channel' generally means that it is resistant to eavesdropping. Your example in this context most definitely means they do not need a secure channel to exchange public keys. The whole point of a public key is that it can be transmitted publicly and shared. The New York Times could publish a public key on the front page, and anyone could use it to send information to them securely. The more public, the more comfortable you can be that you are talking to the right person. If a symmetric key becomes public, it can be used to decrypt any captured communications. Public keys are meant to be public. The more public they are, the more sure you can be that you are talking to the right person.
Say I do the following:
- Sign the email contents and send my public key with every email I send
- Post my public key to social media: Instagram, TikTok, Facebook, Twitter, Youtube, Github, Reddit, Discord, Twitch
- Have my public key visble on multiple websites I create and maintain
- Take out a full page add in the New York Times displaying my public key
- Send out signed and notorized documents to everyone I've ever met with my public key
- Get a tattoo of my public key fingerprint on my forehead, which is visible in any photograph of my face, including polaroids kept by my family members and photos and videos published on their own social media.
So now I send you an email with my public key attached and signed with my private key. You can compare the public key or it's fingerprint to what is publicly available using any of the methods above, and use it to verify the owner of the corresponding private key (me) sent the email. You can then reply and encrypt your reply using my public key and only I will be able to decrypt it.
You can also have someone else who's public key you already have and trust (Carol here) 'sign' a message containing the public key and a message saying it belongs to me. As long as you trust Carol and the method by which you obtained her public key (possibly in the distant past), you can add to your trust level the fact that Carol thinks you have the right key for Alice.
Alice's public key is useless to Bob unless he can prove that the public key came from Alice.
I wouldn't say it's useless. It depends on what you mean by 'prove'. Proving in a mathematical sense that the key came from a particular person is impossible because you always have to put your trust somewhere, even if it is meeting them face to face and having it be someone wearing one of those Mission Impossible masks. You can use it to encrypt a message to the sender. Even if it isn't Alice, it prevents anyone else from being able to decrypt it, such as a rogue sysadmin at your or Alice's ISP.
The more encrypted communication that happen between Bob and Alice, the more confident they can be. If Bob and Alice exchange emails for years and always encrypt them for example, then if Bob has the wrong key, Alice will never have gotten a message she was able to decrypt. Since they are public, Bob can also ask Carol for Alice's key's fingerprint to make sure it matches his own. He can also verify directly with Alice that he has the right key in some way that he has the right key (face-to-face meeting, phone, text, etc.) and not worry about the conversation being intercepted affecting the security of the information he encrypts.
Also in your symmetric key example using Carol, Carol now also has the symmetric key and could decrypt your communications with Alice. The nature of a public key is that there can be a web of people that you can use to verify the public key associated with Alice, and all of this can be done in public for other people to see. That in fact increases security because anyone that sees someone trying to pawn off a fake key for Alice will have a lot of people that know the correct key and will bring attention to the fake.
And Carol does not have to be involved in the communications at all between you and Alice using public key cryptography. The exchange of information can be asynchronous and distant in time. Imagine this scenario:
- Bob goes to high school with Carol, where they exchange public keys in person. They use these over the years to encrypt emails to each other. Bob has Carol sign his public key with her private key to authenticate that it belongs to [email protected]. Bob can provide this signature to anyone that trusts Carol to validate his identity according to Carol.
- Carol and Bob go off to different colleges. Carol is in a sorority with Alice and they also exchange public keys and communicate using encrypted email. Carol signs Alice's public key with her private key to authenticate that it belongs to [email protected]. Alice can provide this signature to anyone that trusts Carol to validate her identity according to Carol.
- Five years after college Carol gets married. Bob and Alice meet for the first time at the wedding and exchange email addresses. Carol is busy with her wedding and doesn't know they even met.
- Alice sends Bob an email from [email protected] with her public key and Carol's signature of it that she obtained 5 years ago in college to verify it is her
- Bob can check the Carol's signature using the public key he received back in high school, and can verify the public key he received in the recent email does belong to [email protected], at least according to whoever has Carol's private key or had it when the signature was generated.
- Bob can now send Alice emails encrypted with her public key that only she can decrypt. He should now send her his public key and the signature Carol created for him back in high school. Then Alice can verify Bob's key using the signature and Carol's key she received in college, and encrypt her replies to bob.
- Anyone can send Alice an encrypted email, but to fake the recipient they would have to collaborate with Carol (or someone else that Alice trusts) to create a signature she will accept.
In this scenario Carol doesn't even have to be aware that Alice and Bob are talking, yet by using her private key to create a signature, anyone else with Carol's public key can use that signature to authenticate that Carol's private key at some time signed that information.
If someone intercepts Alice's email to Bob with her public key, it doesn't affect the security. They can't replace the public key without invalidating Carol's signature. If they try to create a fake signature, Bob won't be fooled because he's had Carol's public key since high school.