Odd Server Logs

Question

I have a webapp running using Django. I recently activated some middleware which is supposed to alert me of 404's where the referrer is within my domain (supposedly indicating a 404 that is my fault).

However, today I got 34 requests for urls which don't exist, all of the login variety. Some examples:

/signup
/user/register
/sign_up.html
/tools/quicklogin.one
/member/join.php
/index.php?do=/user/register/
/index.php?action=registernew
/index.php?option=com_registration&task=register
etc.

All of the 404's seem to be dealing with login, and all have the same IP and user agent:

User agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1;) IP address: 10.122.71.83

None of these url's exist in my codebase. I'm using Django not php, so it's not someone clicking on a broken link (my login page is at /login and my register page at /register) Is this just a crawler that I need to configure better or something malicious?

Answer 1

This appears to be a program running on some script kiddies PC, that is randomly trying out sites to detect what CMS they are running, to see if that system is vulnerable to SQL injection or whatever else, and then either automatically exploit it, or pass the information to the person running the program.

Basically just a script looking for a way into your backend, but as long as your site is secure, there's nothing to worry about.

Answer 2

If you're concerned about the 404's where the referer is specified, just wait until look at the 404's where the referer isn't set! (there will be many more)

The truth is that from the moment you connect your web server to the Internet, bots will try to scan your server for web proxies and other vulnerabilities so they can exploit it and send malware to your end users. This is common and detected by seeing 404 entries in your log.

What you should do is make sure your server is patched when you expose it to the Internet, and patched as new updates are rolled out. These bots are trying to exploit servers that have been neglected or haven't been security hardened.

Also, you should know that referer is not a secure field and can be spoofed. This is likely what's happening. I can see how you would logically assume that this is coming from your own site, but it's not a field that can be trusted for security purposes... though it is helpful to find links that users click on and get sent to a page not found.

Anecdotally your web server logs should indicate the source IP of the user who is initiating the action. If it's an internal address (you mention a 10.x address in our post) then it's possible that the user is on your subnet and they have a virus that's scanning your server.

You should do a google search for the UserAgent string, since many webmasters are looking at their logs the same way you are. One example is here.

Bottom line: If your server is patched, and the links have no relevance to your site, you should ignore them, or block their IP.

Answer 3

Using a tool called OSSEC a famous well know HIDS by trend micro you can effectively not only identify but block these attempts. By routing these attempts to black hole route null. I will show you how.

Adding Active Response ∞

  <!-- Active response to block http scanning -->
<active-response>
    <command>route-null</command>
    <location>local</location>
<!-- Multiple web server 400 error codes from same source IP -->
    <rules_id>31151</rules_id>
    <timeout>600</timeout>
</active-response

Testing new active-response ∞

$ bin/agent_control -L

OSSEC HIDS agent_control. Available active responses:

Response name: route-null600, command: route-null.sh

Now we test the response on one of the OSSEC clients.

$ bin/agent_control -b 2.3.4.5 -f route-null600 -u 083

OSSEC HIDS agent_control: Running active response 'route-null600' on: 083

The above blocks IP 2.3.4.5 using active-response route-null600 on agent 083.

To verify the response was run correctly on the client look in the /var/ossec/logs/active-responses.log for something similar to this,

active-response/bin/route-null.sh add - 2.3.4.5 (from_the_server) (no_rule_id)

Now to verify the addition to route table

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
2.3.4.5 

and also another verification

# ip route get 2.3.4.5
RTNETLINK answers: Network is unreachable