I know we use hash functions to check for the integrity of the files etc... but my question is how can we check for the integrity of the files being downloaded from some server?
Integrity is defined only relatively to an authoritative source which tells what the "correct" sequence of byte is. Hash functions don't create integrity, they transport it. Basically, if you have:
then you can recompute the hash function over the file and see if you get the same hash value.
You still have to start somewhere. Some software distributors provide, along with the software, a "checksum" (or "md5sum" or "sha1sum") file, which contains the hash values. Assuming you got the correct checksum file, this allows you to verify whether you downloaded the right file, down to the last bit; and this works regardless of how you downloaded the possibly big file (even if it came over some shady peer-to-peer network or whatever; you cannot cheat hash functions).
Now this does not solve the integrity problem; it just reduces it to the problem of making sure that you got the right hash value. Hash values are small (32 bytes for SHA-256) so this opens a lot of possibilities. In the context of downloading files from P2P systems, you could obtain the hash value from a HTTPS Web site (HTTPS uses SSL which ensures server authentication -- you have the guarantee that you talk to the server you intend -- and transport integrity -- what you receive is guaranteed to be what the server sent). In the context of exchanging PGP public keys with people, hash values (often called "fingerprints" or "thumbprints") are short enough to be transferred manually (printed on a business card, spelled over phone...).
Digital signatures expand on the concept, but they too begin with hash functions. All digital signature algorithms sign not the message itself, but the hash of the message (which is equally good as long as the hash function is secure, i.e. resistant to collisions and preimages).
For many pieces of software, the providers of the files also provide the hash of the file that allows you to verify it's integrity.
For example, here is an example of the checksums the Fedora Project provides for the Fedora 19 x86_64 ISO.
Instructions are also provided as well if one does not how to use the provided checksums.
Generally, if the files are provided through the same medium as the checksums, there is very little real benefit as an attacker that manages to compromise the download will also have the capability to replace the provided checksum. However, this is very useful in the case of the files being downloaded over an insecure connection like torrents or a CDN. In such a situation, the software provider can provided the small checksum on his server while serving the files through a higher bandwidth medium such as torrents or a CDN.