If the recipient is not using PGP/GPG, then you cannot encrypt the message, because this would require the public key of the recipient (which doesn't exist at this point). You can, however, sign your message, since this only involves your own private key.
When the recipient receives the signed message, they automatically get the unique fingerprint of your key. Note that they cannot simply trust the key, because they don't know if it's actually yours. You'll have to arrange a meeting -- either in person or through a (video) call -- where you confirm that the fingerprint is correct. Then the recipient can install PGP/GPG, obtain your key from wherever you've published it (a public key server, your personal website etc.) and mark it as trusted. This allows them to verify the signature of this mail and all future mails from you.
Of course all of this only makes sense if the recipient is actually willing to use PGP/GPG at some point and go through the above steps. Otherwise the signature is rather useless, because it cannot be verified.
An alternative to PGP/GPG is S/MIME which uses certificates signed by a certificate authority (CA) -- just like TLS. This is more convenient for the recipient, because they don't have to personally verify your public key. It's sufficient if they (or their mail client) trust the CA. Then they can immediately verify the signature. However, for S/MIME, you need a mail certificate from a trusted CA.