Where do suspect requests from logged-in users to my website come from?

Question

I have an ASP.Net MVC 4 web application that is hosted on a Windows 2008 server. The application is secured by a custom ASP.Net membership provider. Only users who are granted permission by the admin have access. The application contains sensitive medical data of individuals.

Recently, I discovered that 60 out of 320 registered users are sending suspect requests to my website. These requests get a HTTP status 404 response, as they do not exist on my site and are inadequate. As an example, this is the subset of URLs that contain 'admin':

    /_vti_bin/_vti_adm/admin.dll
    /admin.php
    /admin/597ea5fc-0805-4b3c-8ab8-3d5320d8683c
    /admin/chgpwd.php
    /admin/default.asp
    /AdminHTML/parse_xml.cgi
    /admin-serv/authenticate
    /admin-serv/config/admpw
    /advwebadmin/adminsettings/browsewebalizerexe.asp
    /advwebadmin/SQLServ/sqlbrowse.asp
    /iisprotect/admin/GlobalAdmin.asp
    /mailman/admin/mailman
    /phpmyadmin/sql.php
    /scripts/admin.cgi
    /shop/admin.php
    /shopscript/admin.php
    /SiteServer/admin/
    /store/admin.php

Little harm is expected from a request like /store/admin.php, as no .php files are present on the server. It still bothers me, because my users are logged in when the requests arrive. That is, sessions and authentication cookies are sent with the requests. So, if a valid request were sent, the server could disclose sensitive information in the response, as the server trusts the requests to be genuine. I use an anti-forgery token to prevent cross-site request forgery, but this will prevent malicious get requests. These are almost as problematic, due to the sensitive data on the site. Also, if the suspect requests are indeed originating form the user's browser, there is no 'cross-site'.

So far, I have been able to confirm that the authenticated requests arrive when the user is actually using the application. That is, the suspect requests arrive among valid requests and my log also shows the users have logged-in. I am now trying to log complete requests, including origin IP-address, HTTP headers and cookies. Not trivial, as my application is behind a reverse proxy server.

In the mean-time I am wondering what could be the origin of these requests, taking into consideration that so many of my users are involved. Could it be:

1. Much present malware in the browsers of the users?
2. Other malware on the clients?
3. Man-in-the-middle attacks?
4. Malware on the web server?
5. Malware distributed by my web application, javascript?

Also, the web application is regularly scanned by a McAfee Penetration test tool. This tool does not log in. I have found out that all URLs in suspicious requests, made from user sessions, are also done by McAfee (205 different URLs, so far). McAfee itself, however, uses many others, as well (3300+). This makes me wonder if the requests from users could get mixed up somewhere in the network, due to caching or whatever, so that I am actually looking at a non-issue. Of course, I have a far bigger issue if requests are mixed up. On the other hand, the requests are time-correlated with the user sessions, not with the running intervals of the penetration tests and maybe the URLs are just the vulnerabilities malware are looking for.

Answer 1

The list of URLs you're seeing looks similar to ones produced by tools like nikto which request list of potentially vulnerable or known URLs. Of course a list like this could (and likely is) usuable by malware as well as testing tools like nikto.

One interesting factor in tracing what's doing this would be, what's the user agent on these requests? Many testing tools will have a custom user agent so that they're activity can be detected, while malware will likely try and make it look like a standard users browser string to be less noticable.

Either way it would be more difficult for the tool to completely match the real user browser string, so matching users "usual" user agents with odd requests which appear to come from that user could help you work out what's going on.

Answer 2

Could it be:
Much present malware in the browsers of the users?

Possibly.

Other malware on the clients?

Possibly.

Man-in-the-middle attacks?

Less likely.

Malware on the web server?

Less likely.

Malware distributed by my web application, javascript?

Possibly.

Most likely, though, it's a bot or scanning tool built for the purpose of probing sites for weaknesses. It could be hosted on a user's computer, or possibly not. That point doesn't really matter a whole heap. What matters is what the bot has access to and what your app is vulnerable to.

The fact that a huge percentage of your users appear to be malicious attackers is a bit disconcerting. If it is actually correct then that means you ought to tighten up your registration process, but I imagine you need to examine your events a bit more closely to determine what's going on.

I'd recommend searching through the actual requests logs (a text file IIS outputs) which will get you the URL, status, user agent and IP of every request (among other things) and you can correlate that with other requests to see if it's really associated with logged-in users.

Answer 3

In the end, this was just an artefact, as I already considered:

This makes me wonder if the requests from users could get mixed up somewhere in the network, due to caching or whatever, so that I am actually looking at a non-issue.

Fortunately, only logging got mixed up. I use log4net. At the start of each request, I specified properties like username, requested URL and IP-address, in a ThreadContext of log4net. From anywhere in code, a log entry can be created, without knowing these properties. However, due to thread agility in ASP.Net, at the time of actually creating a log entry, and the properties on the ThreadContext were read, ASP.Net was sometimes processing another request on that thread than the one to which the properties belonged!

So, thanks for your time and help and apologies for reporting this issue which I misinterpreted completely.

Read log4net Context problems with ASP.Net thread agility for an accurate analysis of the issue, and log4net Contextual Properties and ASP.NET for a neat work-around, which I actually applied.