Recently someone broke into one of our linux servers as a non root user.
An excerpt from .bash_history:
[...]
perl
perl
set +o history
set +o history
set +o history
passwd
sdfsdf
passwd
exit
I assume that set +o history toggles command logging to .bash_history
Is there a way to make it so history logging can't be disabled?
Is there a way to find out what happened if history is disabled?
Bash history won't help you against a semi-competent attacker in the usual case.
There's an alternative: The auditing subsystem. Install auditd and configure it, so it logs for example program executions and other things you configure. In basically logs the system calls that are made according to filters you have created.
Now, if you really want to depend on bash history, you can make it more secure: Recent bash versions can log to syslog instead of a local user file. Then, syslog can be redirected to a remote syslog server if that's imporant to you.
ssh root@yourbox /bin/bash to execute bash without a login, leaving no wtmp entry and leaving .bash_history unmodified (thus nullifying the remote syslog approach). Be assured, an actual competent attacker will leave no stone unturned in covering their tracks.
Oct 8, 2012 at 23:52
it won't really help if you allow executing non-shell commands from the shell. For example - you can omit bash by running perl (or some other scripting language) and call commands from there - it will evade bash logging.
An attacker who actually wanted to hide his tracks would just edit the history file by hand.
You can make it harder for a non-root user to hide commands that she's already typed into bash. But this only requires the attacker to be moderately competent.
HISTFILE variable read-only, then bash will save the last $HISTFILESIZE commands into that file when it exits. You can make HISTFILESIZE (and HISTSIZE) read-only as well and set them to a large value. This is very easily defeated though, for example by using the history built-in to sanitize the history before exiting. And since bash writes history entries when it exits and not after each command, just exit bash with kill -9 $$ and no history will be saved. Zsh can be told to save the history after each command, but not in any way that the attacker can't trivially undo.chattr +a ~user/.bash_history. Only root can remove the append-only attribute from a file. As seen above, this only means that if the attacker makes a mistake, she won't be able to erase her traces.Even if you managed to have all commands from a particular shell logged, this wouldn't be useful for security. The attacker could just emit commands from another application, in a way that wouldn't raise any red flag and be plausibly deniable. (zsh, fish: ok, so he prefers that shell. emacs, vi: so he edited a text file. ssh: presumably he just wanted to connect to another machine. ~/bin/foo: it's just an innocent script, but who knows what it was two hours ago?)
Linux (like most unices) has a simple command auditing system called process accounting that's usually enabled by default: the executable name, time and executing user for each executed command are stored in /var/log/account (the exact location may vary between distributions). You can view that log file with lastcomm. This may help you in your current forensic endeavor, but it can be defeated by a moderately competent attacker (e.g. by hard-linking every executable to /var/tmp/$name/ls) — all you get to know reliably is that there was activity from that user at a certain time.
There are more thorough auditing packages, in particular the audit subsystem that john suggested. Search for auditctl to find a few examples of how to set it up. This is typically not turned on by default as it comes with a performance hit (I don't know how bad this is in practice).
Another simple tool that's (at least partially) default-on is file times. If a file's ctime is from before the intrusion, then the attacker didn't modify it. If you have file access times enabled (this used to be the case by default, but is now often disabled for performance; check the *atime mount options), .
All of that assumes the attacker didn't gain root, otherwise all bets are off. Consider that the attacker may even deliberately have left traces that make it look like she didn't escalate her access by cleverly wiping only the more compromising traces. If you're at all worried, reinstall that system from clean backups (as well as other systems that the attacker may have reached following the initial break-in).
Set all the users shells to the bash restricted shell.
In /etc/passwd set the shell to /bin/rbash (it should be a simlink to /bin/bash), or /bin/bash -r
The Bash restricted shell prevents users from changing shell options like command history.
Also, to supplement the ideas listed above, I would install and configure logwatch to run daily (or more frequently) via cron and email you the results for auditing purposes.
Installation and configuration:
Debian-based Linux type the following command-
apt-get install logwatch
Redhat-based Linux type the following command-
yum install logwatch
To customize logwatch go to /usr/share/doc/logwatch-*/ directory and read the file HOWTO-Customize-LogWatch
Edit logwatch.conf file:
vi /etc/logwatch/conf/logwatch.conf
Or
vi /usr/share/logwatch/default.conf/logwatch.conf
Read man page of logwatch for more information.
