Any public PC could be riddled with traffic analysis malware, keyloggers and other nefarious pieces of software. In particular, if you plug a USB key in the PC and run some code from that key, then you have just demonstrated that the PC is configured to actually allow running arbitrary software from USB keys -- probability that the PC is "clean" is, at that point, abysmally low.
This is true for your PGP keys, but also for any authentication password that you type on the machine. In order to obtain the PGP-encrypted email in the first place, then you must probably connect to your Webmail, which will require a password. If you typed the password, then you're toast.
Personally I never use a public PC to do anything which is not inherently public. I will use such a machine to lookup maps and information Web sites, but not to access my private data, and that includes my emails.
What could work, though, would be a personal computer (say, a smartphone) which plugs through USB on the public PC, and presents some software which, when executed, runs a network proxy for the smartphone, taking care of encoding and decoding packets into some custom USB-based protocol. In essence, this would turn the public PC into a glorified network access point, which needs not be trusted as long as only "secure" protocols are used from the personal computer (SSL, SSH...). I am not aware of a readily available implementation, but it would theoretically work, at least as long as the public PC allows running custom software from an inserted USB key.