Let's say I were to post my e-mail address and public PGP key to my website. When a client contacts me, they would have to include their PGP public key in the e-mail so that I can respond to them in encrypted PGP format, am I correct? Or is there some other way to reply to them without them having to include their PGP public key in the email?
If the correspondent's key is posted to a public key server, e.g. pgp.mit.edu, your email client can probably find it. If not, the correspondent has to to supply it. Read carefully the answer by fgrieu concerning trusting keys supplied in message from another. It's not safe.
(You should consider posting your public key on a key server, too, and read about PGP/GPG key signing: https://www.google.com/search?q=pgp+key+signing because a signed key is less susceptible to attacks that replace your key with another's.)
You are correct that you need your client's PGP public key in order to use PGP to either
- send to your client confidential information ;
- confirm that information allegedly from your client is actually from that source.
But it is questionable that you can safely get your client's PGP public key using only the procedure considered:
When a client contacts me, they would have to include their PGP public key in the e-mail
Problem is, you do not know for certain if the email you received contained your client's PGP public key, or a public key apparently from your client (with your client's name or/and email) but actually made by Eve impersonating your client, and Eve the only holder of the corresponding private key.
One way to confirm that a PGP public key belongs to someone is to cross-check the key's fingerprint in a face-to-face meeting; or, by extension, by phone. Another is to to check that the public key comes with signature certifying that the public key actually belongs to who it appears to belong to (as identified e.g. by the name and email in the public key), the signature being made by someone/something for which you already have a trusted public key, and who you trust to make correct decisions when signing PGP keys; PGP makes this relatively easy, see PGP web of trust.
Note: if your client is defined as whoever actually prepared that initial email's attachment, then your procedure is indeed secure.
Note: As to an alternate mean to get your client's alleged PGP public key (using a server), NOT solving the above confidence-in-the-key issue, I refer to this answer; it also gives the valid advice to try and have your own public key signed by reputable parties.