I had a VM based on Ubuntu 16.04.03 and the only package added to it was OpenVPN. The purpose of it was to provide a VPN into my private network sited within the cloud environment. The hypervisor is also 16.04.03.
The other day I noticed 100% CPU utilization on it and once I logged in I noticed a "yam" process consuming all the user cycles. Google search revealed it likely to be a bitcoin miner.
None of the other VMs on the system or the hypervisor seemed to be affected. I have 20 hyperthreaded cores on that system so I reasoned that if the malware had been capable of it they would have gone after 40 threads instead of the 2 they got.
So I killed the process then took down the VM with the intent of rebuilding it in the morning. Foolishly I didn't take other actions.
In the morning I discovered that all the VMs on that system were down and the passwords had been changed on the hypervisor. I could log in via an SSH authorized_key authentication but could not sudo with my regular password. Worse, the ZFS zpool had been destroyed. All the Ubuntu updates got applied.
So I had to go to the co-hosting location site (fortunately local) and recover the system with console access. I was able to bring up the storage array and the other VMs. I changed all the passwords (quite a lot of them quite a pain.)
So my questions are as follows:
What exploit might have been used to get into the VPN VM but not the others? Only that VM had OpenVPN on it.
What exploit might have been used to take down the ZFS array and passwords in the hypervisor? Is it possible the exploit could have been closed by the upgrades?
It seems to me that if they had root-shell access they could have done a lot more damage than they did. Is the fact that they did not a) because they were being "nice" or b) they didn't really have that level of access?
I realize that these questions can't be answered definitively with the information detail I have here, but any guidance or suggestions from someone who has experience with a similar situation would probably be helpful.