Do email security scans that follow links replace digest params in URLs with invalid values?

Question

Going through our application logs recently I noticed a number of instances of page requests where an invalid SHA512 digest param in a URL resulted in a 403.

A couple aspects of these requests make me suspect an email scanner is checking our links for malware, as described in this answer. First, the requests tend to occur within a few minutes of the email with the link being generated. Second, the requests appear to come from the same IP range.

Whatever the source, it definitely appears automated. The changed digest gives me hope that this is a friendly email scanner trying to avoid causing problems for our application by changing the digest to prevent the request from authorizing and taking some action within the application.

Do scanners do this sort of thing? Is there any documentation of this? I’m hoping what I’m seeing is evidence of an email scanner and not something malicious.

Answer 1

I don't know any antivirus that would modify URLs before checking them. Nevertheless, if there is such an antivirus, such approach would make little sense. The attacker can create a harmless site that is only used for redirection. If an ID in the URL is invalid, there will be no redirection to a harmful website and the antivirus will get just a harmless "not found" page. Thus the antivirus will classify such link as not harmful, which will be wrong.

It is a bad practice to use links that have immediate effect on some data, e.g. when visiting a link leads to registration confirmation or to unsubscription. The attacker can register a user at some malicious website and by visiting a confirmation link the antivirus will confirm registration without user knowing about that.

I would discourage you from using such kind of links in your emails.

Any change related to the user data should require explicit action from the user. The page loaded by the email link should contain a button that needs to be clicked by the user to trigger the needed action. This action should use HTTP POST method to prevent antivirus from traversing further links via GET and triggering undesired actions. Then, if antivirus opens such URL, no action will be triggered and user data will remain unchanged.

Answer 2

Consider the scenario where an email message contains an unsubscribe link, or a poll link (e.g. "how do you rate our service") - all requiring a single "visit" or "click" to be accepted. A solution that checks the safety of links would trigger these, and the recipient would be unsubscribed (or would have rated poorly the service of his favorite pizza joint, for example).

That's why some of the "safe links" types of services take this approach - testing the link with randomly generated (or not so random) parameters to see if pages would still render correctly, or if they look the same. This is generally done by smaller antivirus vendors (not by ProofPoint, Google, or Microsoft).

This is a field of intense research, where AI would be extremely useful, as there's no "foolproof" method to resolve the issue of fake clicks (generated by automated link verification services).

I also recommend seeing this StackOverflow discussion on the subject.