I am trying to figure out how it was compromised. They installed IptabLes and IptabLex in /boot
.
They also added /etc/init.d/IptabLes
and /etc/init.d/IptabLex
which simply call the respective /boot
files. It seems this attack uses a lot of bandwidth (probably a DDoS); I noticed it immediately.
The server is running CentOS 6.5 with all the latest updates.
It runs logstash
, redis
, ElasticSearch
, and Cherokee
webserver serving Kibana.
I am thinking it must either be ElasticSearch or Cherokee web-server.
ElasticSearch port (9200) was open to the world, because Kibana requires it to view the nice graphs. Redis ports (6379) were restricted to only 5 known hosts via iptables.
Cherokee webserver runs on port (8080) not default of 80 and was open to the world.
SSH does not seem to be compromised. The server uses keys and no password authentication
is allowed. We run SSH on port 2020, which is listed as (xinupageserver) in iptables.
Here are the iptables rules. Notice redis is restricted to web hosts,
but http via Cherokee (webcache) and ElasticSearch (wap-wsp) are open.
➜ ~ iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere
REJECT all -- anywhere loopback/8 reject-with icmp-port-unreachable
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:xinupageserver
ACCEPT tcp -- web1.mydomain.com anywhere tcp dpt:6379
ACCEPT tcp -- web2.mydomain.com anywhere tcp dpt:6379
ACCEPT tcp -- web3.mydomain.com anywhere tcp dpt:6379
ACCEPT tcp -- web4.mydomain.com anywhere tcp dpt:6379
ACCEPT tcp -- web5.mydomain.com anywhere tcp dpt:6379
ACCEPT tcp -- anywhere anywhere tcp dpt:wap-wsp
ACCEPT tcp -- anywhere anywhere tcp dpt:webcache
ACCEPT icmp -- anywhere anywhere icmp echo-request
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
Chain FORWARD (policy ACCEPT)
target prot opt source destination
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere
Thanks so much for the help.