Is checking for reputation of a URL a good approach?

Question

Saw some document such as this that mentions the checking of reputation of URL as a means of detecting malware. I feel that this is something that can be easily exploited by hackers:

When a hacker plans to set up a malicious URL, he will first make the URL benign and ask online services such as virustotal to check its reputation, of course it's all good. Then he changed the code for the web app (say, a php page) to be malicious.

Wonder if this possibility makes the URL reputation checking less useful.

Answer 1

The fact that a cracker could change the content on the page afterwards is not such a big problem. Virustotal (and many others) allow you to rescan the URL instead of checking the old report.

The real problem here is how, not when these sites determine if a site/page is malicious or not:

• Looking for signatures of known malware is very unlikely to catch unknown malware.
• The server could be cloaking and show a clean version of itself to the scanner.
• Analyzing the behavior of a page won't help either, it could detect that something's off, or it could require some form of user input. Or in the most extreme case: it could be a time bomb that will go off just one minute after having been scanned.

Scanners will only catch the common flu of the WWW, not something that really is intentional. If someone wanted to be happy, all he'd have to do is to be creative and do something that hasn't been thought of before.

If you want to be on the safe side, I can recommend some browsers that are pretty secure: a telnet client (it has poor TLS support though) and GET from lwp-request.

Answer 2

Checking the reputation of a URL is a useful approach but of course does not cover every malware or phishing URL. URL based filtering is very fast when compared to the analysis of the response payload. This allows for an early and cheap blocking of the access to potential malicious resources and thus might also preserve more resources to deeper analysis of yet unknown sites.

Reputation is commonly not done based on a specific URL but on the site, i.e. if there is a single definite malware infection on this site then the whole site (or sometimes even the IP address) gets a bad reputation. This makes sense because if the attacker manages to have one URL on the site deliver malware then chances are high that the attacker is able to control more than this one URL.

Reputation services can also not simply be tricked by letting somebody like virustotal check the URL or a even the whole site. Instead these reputation services scan the web themselves and also get user input or information about bad sites from analyzing phishing or spam mails. They also include historical information about domains, i.e. how old the domain is, how often the IP addresses have changed in the past etc. One widely used reputation service is Google Safe Browsing which is enabled by default in Chrome and Firefox.

This does not mean that attackers don't try to work around these reputation services because they actually hamper their business. One way is to hack sites with a good reputation and use these for malware distribution. For example Jamie Olivers website was hacked multiple times in 2015 and used to spread malware.