I assume we want a deterrent against software-only attacks targeting the passphrase, unauthorized use of the private key to decipher or sign, or unauthorized access to deciphered data (including emails).
It would be effective to:
- Disconnect from the internet
- Make a full backup
- Use PGP/GPG with private key and passphrase
- Fully erase the machine (can be combined with 5)
- Restore the backup
But without 2/4/5, disconnecting from the internet is only mildly effective. In particular, a software keyloger targeting the passphrase, or malware keeping track of deciphered data, has a fair chance to do its thing then exfiltrate the result after 6.
Problem is, 2/4/5 is a pain.
One reasonable option is to perform 1/3 in a Virtual Machine, and destroy it afterwards (or equivalently, restore a snapshot made before 1).
Booting on a live CD (assumed malware-free) is a nice option, but since there is no Windows live CD AFAIK, this won't accomplish much of what a Windows user want to do if HD storage is encrypted, or data (like emails) in a Windows-only format (Outlook..). The only reason I see to disconnect from the internet on top of that is to prevent exploitation of a vulnerability on the live CD.