It depends on your definition of "compromise". The main concern is privacy, for which parcimonie and tor are solutions, as noted below.
But first lets look at data integrity.
The notion of "end-to-end" security is most helpful when it covers the actual application you have in mind for the data. Since the objects moved around by the OpenPGP HTTP Keyserver Protocol (HKP) are covered by signatures based on public-key technology, they can be checked by your application (e.g. gpg) itself. It will establish whether the signatures are valid, whether they apply to other keys and people you trust, etc. (Update: So gpg only (optionally) gets the certificates from the keyserver, it does not rely on any keyserver to establish the validity of the keys.)
For example suppose you're using gpg to check signatures on email. If gpg tells you a given key is not known, you fetch it from a keyserver. Perhaps your mail then tells you the key is known but not trusted, since there isn't a trusted path (e.g. leveraging wotsap) from your ultimately trusted keys to the one that signed the email (like this one). So you look at the signatures, and find the key of someone who is trusted by someone you know, and who trusts the given key and download that too. If you end up with a path you trust, everything is great. If someone mucks with the keys while you're downloading them, you either won't get a trusted path (due to broken or unhelpful keys), or (and this would be pretty unusual) you'll get another path that might or might not work for you. The point is, though, that you still have to use gpg to check the paths, and evaluate the "intangibles" of the trust path yourself (how much you trust people and their understanding of signatures, etc). But you always have to do that with PGP. It is thus "end-to-end" in the sense that it is secured all the way from the person who made the signature, to your software which checks the signature, and your configured trust roots, and is thus pretty much ideal.
The security of the transport is not particularly important, and using TLS and the associated PKI and certificates would add extra complexity, cost and brittleness to the PGP keyserver infrastructure, without improving data integrity. If someone messed with the transport, they could deny you service, or do something like a phishing attack, but you nearly always have to deal with those sorts of issues anyway.
See e.g. the "Security Considerations" section of the (expired, but presumably still useful) Internet Draft on the subject: draft-shaw-openpgp-hkp-00 - The OpenPGP HTTP Keyserver Protocol (HKP)
A much more significant data integrity concern would be the risk of being fed a carefully crafted "malware certificate" that can exploit a buffer overflow vulnerability in your gpg client, or something like that. I haven't heard of any such exploits in gpg, but they have happened for SSL: yaSSL Certificate Processing Buffer Overflow Vulnerability - CNET. This is explored more at What risks are inherent with connecting to an untrusted public key server?
A more problematic concern is privacy. If you don't want people to be able to see what keys you are searching for, or to mess with your key transfers, using TLS would be an alternative. But it doesn't protect you from the server knowing what you're interested in.
@teris-riel expands on the privacy concern: Imagine someone sends you a signed then encrypted email. Your mail client happily retrieves the key for the signature. Evan is watching the traffic and sees the key fly by. Now he knows 1) that you can read messages encrypted with the key he sent to, 2) that you did read that particular message, and 3) the approximate time you read it. If this kind of confirmation attack worries you, use hkps and Tor to get keys, and don't do it automatically. parcimonie is a tool that works to solve this problem.
And as a parcimonie implementation in bash notes: gpg --refresh-keys discloses your entire list of PGP keys to the keyserver you are using, as well as whoever is wiretapping your connection if you are using an unencrypted protocol such as HKP (which is the default for most setups). That is a bad thing.
So use something like parcimonie.
There are some alternative protocols, but finding servers for them which contain the keys you want seems hard:
hkps://for the protocol.