I understand that application whitelisting is better and more accurate than black listing. However, my question is, if you whitelist a directory, won't the malware also be able execute in that directory? So what use is application whitelisting in this respect?
If you whitelist a directory, then sure, any malware in that directory can run.
That's why you don't whitelist directories. Instead, you whitelist programs, and you take precautions to ensure those progrms aren't modified. For example, instead of whitelisting "c:\Program Files\Internet Explorer", you whitelist "c:\Program Files\Internet Explorer\iexplore.exe with SHA-256 checksum c09bc04058f1e2d4eae481490b998381486311e02ff782e99383c16d77c1b3bc".
In a corporate environment the purpose of whitelisting is that an administrator can authorise programs to run, but a regular user cannot.
In that case it is ok to whitelist a directory that a regular user does not have write access to. In fact, there is a standard AppLocker profile that allows execution from C:\Program Files and C:\Windows, but excludes user-writeable directories. It is rare to see this in practice, but I think it is an excellent security control. In particular, it stops a user downloading an exe file from the Internet, and running it.
The approach Mark mentions of hashing individual exe and dll files, while nice in thoery, has been generally discredited as impossible to implement in practice.
I can imagine your follow-up question: what if malware does privilege escalation and writes itself into a whitelisted directory? Well, clearly that malware will then run again in future. AppLocker white listing does not protect against that. In fact, very few protections work against malware that has escalated privileges to root/administrator. Whitelisting is a useful technique - but it is not a silver bullet.