Tracing origin of malicious PHP script

Question

I'm looking at a Linux server running Apache + mod_php. A malicious PHP script was uploaded during the night. It's owned by the Apache user and I would think it almost certain that the script was generated by an exploit in some of our existing PHP code.

I looked in the Apache log for hits on the script, and from that got the attacker's IP. I then searched back for other activity by this IP ... nothing. I also searched back for hits from the same User Agent (in case the attacker was connecting through proxies). Nothing.

So I then grepped the log for POST requested issues in the few minutes before the first hit on the malicious script, in the hope of finding which legitimate script the attakcer had exploited. A few hits, but nothing that stood out. For each of these scripts I've appended some code at the top to dump POST data to a log file.

I then searched the Apache log for strings like 'lynx', 'wget', 'tmp' in case the attack had been via GET. Nothing.

So I'm a bit stuck now. I wonder if anyone can think of any clever ways to determine how this malicious script was uploaded, or any additional logging I could put in place? (mod_dumpio is an option)

Answer 1

Let me answer with a few observations, and comments. I will begin with the "whodunit" approach of aiding in determining who, what, when, where, and how.

• What - a file you found on your system
• When - what date was it found
• How - how was it uploaded
• Who - who uploaded it

You already know the file because you found it. Let's call this file: malicious.php. What you mention doing is determining "who" accessed this file, but you need to figure out how it was uploaded. Looking at the who accessed this file is the wrong approach. This is because attackers usually use multiple attack points. E.g., one system may scan you, another may compromise you, and yet another may access the system one compromised. So let's figure out the who first using deductive methods:

$ ls -ltha malicious.php 
-rw-r--r-- 1 www-data www-data 0 Mar 27 10:59 malicious.php

In this instance, we see the file belongs to user www-data in group www-data. This does not mean an attacker uploaded it via http using post or get. There is nothing to stop someone from exploiting say telnet, uploading a file and performing a chown on the file.

ls -ltha --time=atime malicious.php 
-rw-r--r-- 1 www-data www-data 0 Apr 27 10:09 malicious.php
ls -ltha --time=ctime malicious.php 
-rw-r--r-- 1 www-data www-data 0 Apr 27 11:00 malicious.php

We can see the differences in timestamps here. If someone used say timestomp (Metasploit) MAC times can be changed. So we can deduce on the data above, that we are more or less within the 50 minute realm. Let's ensure that this user has never logged in, for that we can check wtmp, auth, ftp logs, etc. If this user has never logged in, we can then infer that this was not uploaded in any other fashion outside of via HTTP.

When - we reduced the timeframe to a degree to a 51 minute period. The logs I would look at would be error logs FIRST followed by access log. The reason for this is simple, at no point in time would an attacker know exactly how to get a file on your system. This means, there was a trial and error recon. This will be in your error logs appearing as a 404 or 403. I would search those first. Now let's further reduce the searching. A log analysis of visits will show what is normal versus what may be an anomaly. For example, if the base of traffic goes to say index.html remove all those instances, and look at the anomalies. One you determine this, you have then determined the How. This is completely separate from figuring out how they compomised your your code which would be a completely different question.

Answer 2

Kudos for finding it quickly, it looks like you're doing something right. But you're also doing a lot of things wrong. The most obvious one is that directories within your document root are writeable by the webserver UID.

It would help to know what you are trying to achieve by "tracing the origin" of the script. Certainly you should be looking for the vulnerability which was exploited so you can plug the hole, but anything else is merely curiosity.

grepped the log for POST requested issues in the few minutes before the first hit on the malicious script...I'd already looked at the timestamp on the file

So you used the mtime on the file to locate the log entries and then tried to use the log entries to determine the creation time? Somewhat circular logic here.

How many log entries do you have 1 second either side of the file's modification time? How many unique PHP scripts does that resolve to? Certainly you can't rely on the mtime being an accurate representation of the creation time - but it's a start.

As a minimum you should be checking all your PHP scripts for $_FILES, and include/include_once/require/require_once/eval with non-literal arguments.

Ast to what logging you could add....you don't say what logging you currently have in place. In addition to preventing the weserver from writing anywhere in the document root, setting up inotify to report any changes to files. Not to mention the usual steps in hardening a PHP server.