When you cipher your mail (or any other kind of message or file) using your private key, and distribute you public key, this will guaranty that you are the actual sender of the message and that it's content has not been modified (otherwise the decryption using your public key would fail). This ensure message authenticity.
To ensure privacy, you will need to encrypt your message using your partner's public key. Then, only him can would be able to decrypt it using his private key.
By this mean, by ciphering using your own private key and your partner's public key, you ensure both privacy and authenticity of your message. And, as you said, in order to work, such system needs for you public key and your partner's public key to be freely available.