If you do the job in a right manner, no:
All this is public, shareable and signed and fingerprinted.
Your provider (or Charles) could even see that you have been exchanged some keys, but could'nt know which one you have really added to your keyring (or even if you add one).
For sample, you could download the entire public keyring from one keyserver but store no key or only one...
If you verify the fingerprint given by third party (by an alternative, trustable way, like paper, phone, fax, sms etc...) of any public certificate,
the transport make no matter.
The best recognized way to understand this and grown you personal network, is to participate to key signing party where people exchange fingerprint, showing his personal official passeports and back to home, each of them sign the key of each others...