In the X.509 model, there is no restriction on download. When a verifier (say, a Web browser) wants to validate a certificate, it will try to obtain extra information through other objects: certificates, CRL... and it may obtain these objects over insecure channels. That's the point of CRL being signed: a verifier will trust a CRL not because it just obtained it from a specific Web site, but because the CRL is signed by an authorized CRL issuer (usually the CA itself).
The bright side of such a principle is that it makes X.509 certificates applicable to all kinds of networks. Otherwise, you would have a rather harsh chicken-and-egg problem: how would you trust that you got the right, most recent revocation information ? Because you downloaded it from a HTTPS server ? But how did you verify the certificate of that server ? And so on...
But there is a dark side, of course: a CRL is a snapshot of revocation information at a given date. Being signed, it is immutable. It cannot be changed. It is thus always a bit stale. A verifier will require that the CRL it uses are "not too old" -- and that, inherently, implies regular publishing of newer CRL.
On the technical point of view, we may envision a specific kind of CRL which does not contain the whole list of revoked certificates (which can be bulky) but only a short message stating "no certificate revoked since that date", thus allowing to somehow extend the lifetime of a CRL. This exists; it is called delta CRL. With delta CRL, a new CRL is virtually emitted at regular intervals, without necessarily forcing the download of a new big file.
Unfortunately, not all software out there know how to use delta CRL.
Another technique is segmentation: split the CRL into many smaller files, each talking about only a subset of certificates. Down that road lie very small CRL which assert the revocation status of only one certificate at a time; this is known as OCSP. There again, support is in deployment.