Analysis of hacked servers — daily RDP connections I can't find an obvious reason for

Question

I'm cleaning up after an incident where a few servers were hacked. A new IDS has been put in place, the firewall has been upgraded, the services used in the attack are being replaced with more secure alternatives, and everything that doesn't need to be exposed is now behind a VPN, so at this point my focus is on determining what was going on while these servers were hacked. This is interesting to me and I wonder if any of you have seen something like it.

The hack started with RDP brute force and created a second account and then spread over RDP as far as it could using the same credentials and whatever it could dump from the first server. Then, for a period of several months, the hackers connected a few times a day over RDP for anywhere from a few seconds to a few minutes on both of those accounts. To try to figure out what might have been going on, I've:

• Run sleuthkit against the drives
• Written a tool to sequence the file timeline against all useful event logs and group those by remote session intervals and also run other profiling on the sessions
• Reviewed the users on the box
• Scanned the box
• Looked at all services listening on any port
• Manually gone through the files

I haven't set something up to listen for odd traffic to or from these boxes since we've taken them offline and I'm working with them as disconnected VMs.

Apart from some signs one of the boxes might have been temporarily used to stage a social engineering attack, I haven't seen any activity of note during the entire time the boxes are actively hacked. I don't imagine a higher form of access is taking place since the RDP connections were so active, and based on the firewall and services that were still working, the only form of backdoor I can think of actually working without detection would be a reverse shell, and if that existed, I'm not sure why RDP would be so active.

It turns out that one of the servers hacked by the hacked server had also been hacked the same way several months prior (by IPs from the same region), and that exhibited the same sort of usage: daily short RDP connections from the same set of sources after the source rotation from the brute force attacks had finished, but nothing odd on the box itself.

I suppose the continued connections could be just validation of current account credentials and access levels, but I'd have to imagine more would have been going on that I'm not seeing in the logs or file audits. Does anyone have any related experiences they'd like to share lessons learned from?

Thanks.

Edit: As requested by @pacifist, here are the logs I have reviewed:

• Sleuthkit file timeline
• Microsoft-Rdms-UI/Admin
• Remote-Desktop-Management-Service-Admin
• Remote-Desktop-Management-Service-Operational
• Microsoft-Windows-TerminalServices-SessionBroker-Client/Admin
• Microsoft-Windows-TerminalServices-SessionBroker-Client/Operational
• Microsoft-Windows-TerminalServices-RemoteConnectionManager/Admin
• Microsoft-Windows-TerminalServices-RemoteConnectionManager/Operational
• Microsoft-Windows-TerminalServices-PnPDevices/Admin
• Microsoft-Windows-TerminalServices-PnPDevices/Operational
• Microsoft-Windows-RemoteApp and Desktop Connections/Admin
• Microsoft-Windows-RemoteApp and Desktop Connection Management/Admin
• Microsoft-Windows-RemoteApp and Desktop Connection Management/Operational
• Microsoft-Windows-TerminalServices-SessionBroker/Admin
• Microsoft-Windows-TerminalServices-SessionBroker/Operational
• Microsoft-Windows-TerminalServices-TSV-VmHostAgent/Operational
• Microsoft-Windows-TerminalServices-TSV-VmHostAgent/Admin
• Microsoft-Windows-TerminalServices-ServerUSBDevices/Operational
• Microsoft-Windows-TerminalServices-ServerUSBDevices/Admin
• Microsoft-Windows-TerminalServices-LocalSessionManager/Operational
• Microsoft-Windows-TerminalServices-LocalSessionManager/Admin
• Microsoft-Windows-TerminalServices-ClientUSBDevices/Operational
• Microsoft-Windows-TerminalServices-ClientUSBDevices/Admin
• Microsoft-Windows-TerminalServices-RDPClient/Operational
• Microsoft-Windows-TerminalServices-Licensing/Admin
• Microsoft-Windows-TerminalServices-Licensing/Operational
• Microsoft-Windows-TerminalServices-Gateway/Admin
• Microsoft-Windows-TerminalServices-Gateway/Operational
• Application log
• Security log
• Setup log
• System log
• IIS logs
• Firewall logs
• CAPI2 and BITS logs
• Powershell log

I've reviewed every other log file in the event viewer, but most didn't give any more information than what was already in the above or were disabled. However, the log retention policies weren't set properly until just recently, so there are definitely gaps between the log files.

Answer 1

I have experienced something very similar.

A while back we had a similar incident. A host running RDP on a non-standard port exposed to the internet was compromised by brute-forcing bad credentials that were associated with an old test account that no one ever disabled. Once the attackers gained access to the machine they did the same thing you are describing where they would login for a few minutes once or a couple of times a day then they would drop off. The people had access for about three months and from what we could tell after extensive review of our network they did not try to access any other machines. I spent some time looking through the files in the user profile, since it was a test profile it was never really used so all of the data was from the attackers. There was not much a few images and that about it nothing that was malicious.

After about three months of the same people connecting (I say same because it was the same few IP's in Nigeria that kept connecting), some new IP's originating from Europe and America's west coast started to show up and they set up a spam bot and did some basic testing with it. Once they ran the bot full tilt we caught on pretty quick obviously but we had a lot of discussions internally on how it was strange that some people had access for about three months at a time and only ever connected for a few short minutes a day.

In the end, we came to this conclusion based mostly of speculation. It looks like the original people compromised the server and then logged into check the credentials each day because they were trying to sell access to the machine. Once they sold the access the new people came in and used it as a spam bot. Once they did that we discovered them and shut it down. Like I said this is pure speculation based on the facts we were seeing, but it is our best guess as to why we saw the patterns we were seeing.

The attack we saw was not very sophisticated. The attackers brute-forced their way in and then set up a spam bot and did not try to hide it at all. At the time there was not much logging at the site and no real good way to look at what happened in the past besides looking at basic logs. There was not a ton to go off of but after looking at what we had and doing an extensive investigation similar to what you are describing this is the conclusion we reached.

After this we made sure to clean up our Active Directory to tighten controls and make sure that old accounts were cleaned up etc. We enforce good passwords and enabled more advanced logging. We have also started to implement an MPLS solution between our various sites so that we can greatly restrict the number of services exposed to the internet. We have also improved our anti-virus coverage and configured it in a way that protects our machines and alerts us to suspicious things when they happen. Since this it has greatly improved our visibility in our systems and helped to reduce the number of issues.

Hope this helps.

Answer 2

what logs do you have?

This could be anything, right up to a source of data exfiltration which can be implemented different ways - the neatest I've seen was a library someone made to have the equivalent of an animated qr-code type powershell script where even with RDP or citrix sessions that are fairly locked down someone could still transfer out large amounts of data that get converted back from the qr-code like output.

It sounds like your incident goes back a long time-period - but your thread seems oddly timely with the RDP vulns that have been popular this week.

Answer 3

You can also use RDP cache. This open source 'RDP Bitmap Cache parser' processes bcache*.bmc and cache????.bin files found inside Windows user profiles.

That cache is storing sections of the screen that infrequently change so there is a chance that some malicious activity was recorded.