It's secure, but not perfect
PGP alone ensures privacy, but how you use it may introduce additional vectors for attack, which may compromise security.
The problem lies in the fact most email readers render HTML and that HTML can be exploited. PGP doesn't prevent the SMTP subsystem from being exploited unless you can safely extract the PGP MIME data.
Specifically, PGP messages as you describe are often not scanned for Viruses, Phishing, Malware, or other scams at the gateway. More often than not, the technology to protect the desktop user isn't as sophisticated, or out of date.
Since the SMTP body can have additional alt-text representations, and it's possible that the main text to be displayed may have an HTML exploit injected into the body. This can be done by any malicious relay, or hop in the middle.
To limit that exposure, I would recommend DKIM signing from a domain you control and forwarding that out a trusted MTA. Make sure the appropriate options are set (no
-l setting, and do sign the header, subject and body)
Although a VPN and TLS for SMTP would help protect your message, it is a point-to-point security mechanism where all trust starts and ends between two nodes of the communication. (your MTA and Microsoft's MTA).
An improved solution
Any of the following: PGP, S/MIME, GnuPG
DKIM sign the message, on a TLD domain you control.
Use antivirus and antispam countermeasures on the client, and make sure the client validates the DKIM blob before opening it.
Use a VPN if you're sending email over a Wifi connection