26

The idea would be to prevent an attacker who has stolen a root/admin account or escalated to clear his own activities or even read the traces of what he is doing. Let's assume we are under Linux, we log with auditd, have centralized logs, and we can use MAC with SELinux. But I am interested also for answers under Windows.

One solution would be to forbid all root accounts to access the logs. Logs are managed only by authorized processes on specific servers from logrotate, syslog, and all SIEM stuff. So only the SOC can read and analyse the admins' logs. Only a purge process can delete old logs. Can anyone confirm this is doable?

Is it possible to have something more flexible where admins with their own root privileges could read the logs of other root accounts?

Improve this question
8
  • 14
    The generalized version of the answers: any log should happen on a location which is unreachable for the logged entity. In the case of a linux roots, it means mainly remote logging to other servers. So he will be able to turn off the log, but not to hide its trace before that (including the logging turnoff), except if he is doing it very smartly.
    – peterh
    Sep 21, 2018 at 21:39
  • 3
    *.* /dev/lp0.
    – R.. GitHub STOP HELPING ICE
    Sep 22, 2018 at 20:33
  • Windows does have a version of RBAC. I'm not a windows guy, so you will have to research the details yourself, but it might be worth checking this as an alternative.
    – Tom
    Sep 24, 2018 at 8:31
  • I agree logs shall be centralized. Regarding my flexible request, what about retrieving or copying each admin event provided by auditd in a log file dedicated to this admin account and manage access to these log files with a policy preventing one admin to access his logs with help of SELinux? Retreiving or copying would be performed by say a 1' cron job.
    – lalebarde
    Sep 25, 2018 at 9:42
  • With your addition of the fact that you already have centralised logging, the rest of the details do not make as much sense.
    – schroeder ♦
    Sep 25, 2018 at 15:51

7 Answers 7

Reset to default
44

The accepted solution to this is to not store the logs locally, but on a log server. Once the logs are there, you can restrict or limit access as you see fit.

In some log server/aggregator solutions, you can limit a user from seeing entries that contain references to certain data (like their user accounts or machine IPs). This means that you can enable admins to see other admin activity, but not their own.

You typically want to place alerts within the log server/aggregator to trigger if logs from any one machine stop coming in or are reduced below certain thresholds, which helps to detect if a local admin has prevented the logs from being shipped to the log server/aggregator.

Syslog servers, SIEMs, log aggregators, ELK stacks etc. There are numerous options for you to explore.

Improve this answer
8
  • 1
    My favorite old-school solution. Fill up the disk on which logging is done. Unless that disk is several tens of TB I'll have it filled with dummy logs in short order.
    – Joshua
    Sep 22, 2018 at 2:47
  • 25
    I thought the old school solution for secure logs was a line printer...
    – Paul
    Sep 22, 2018 at 3:49
  • Of course, this just raises the question "How do I partition privileges in a distributed system?" which is a far more interesting one IMHO.
    – Kevin
    Sep 23, 2018 at 16:50
  • 1
    @Joshua Logging software often throttles excessive triggered logs and prioritizes emergency or alert-level messages. It'd be hard to fill up the disk just by spamming some action that creates a log (e.g. writing to a deprecated procfs entry), and unless you have root, you won't be able to fill up the entire disk anyway, since some extra space is reserved for daemons (like logging daemons) running as root, at least on certain filesystems.
    – forest
    Sep 24, 2018 at 2:42
  • 2
    @Paul although Cliff Stoll had to make do with character printers and terminals
    – dave_thompson_085
    Sep 24, 2018 at 3:20
12

Any logs on a compromised host are suspect. You need a centralized logging platform, either a central syslog server/ splunk / logrhythm / whatever. Keep a different set of administrators and accounts. That's the whole idea.

Once you get a platform in place you can delegate the rights to view their actions, either their own or other admins - can be performed. We had rights read specific log sources and hosts delegated out.

Improve this answer
2

If an attacker acquired high privileges on a machine, then the whole machine is untrusted anymore, let alone log controllers.

A remote logging server is the only option here, though details may vary. You'll be able to handle logs and manage access controls in a safer way than storing the logs locally.

Improve this answer
1

First step is to send the logs to a different server, so that after the first server is compromised (and thus their logs could be altered), that log server would have those log entries. And this server can (should) be protected more tightly.

Nevertheless, someone must be able to administrate that server, if only so that the server can be updated!

Ways you can additionally protect the logs in that server:

  • Require multiple admins for any (non-trivial) action (eg. commands introduced require the confirmation from N admins)
  • The logs can be stored linking to earlier ones in a Merkle tree, so that removal or modification of an entry would affect any later ones¹
  • The hash of the current log could be sent to an external party (getting from a time stamping server, sending a partner company, or simply sending to a twitter feed²).

¹ When arguing to use this it may sell better to state that this uses blockchain technology.

² Or if the logs itself are not confidential, you could even push some log server events in the clear: "jdoe logged into the log server and run rm -rf /"

Improve this answer
2
  • 1
    An often cited method to publish the log hash nowadays is to publish the hash in an existing public blockchain that allows arbitrary message like Bitcoin. That'll make the log as tamper proof as the blockchain itself. Though do note this still only prevent undetectable tampering, but it wouldn't prevent log destruction. For that, you'll need proper backup.
    – Lie Ryan
    Sep 23, 2018 at 2:48
  • What are the concrete tools please behind your solutions, especially the multiple admins and Merkle tree ones ?
    – lalebarde
    Sep 24, 2018 at 13:49
1

Some possible ways to protect logs from the administrator:

  • remote logging to a machine that's not controlled by the administrator in question
  • write the log to a write-once media (e.g. Bluray Recordable), note you may also need a strategy to prevent overfilling the media, such as disconnecting all admins if the logs are about to get full
  • publish the hash of the logs to a public blockchain, this detects tampering, though not log destruction
  • force access to the machine via an mitm server, admins can only connect to the administered machine through the mitm server, which will record all key/mouse inputs and screen/terminal output (using either ssh mitm or vnc/rdp mitm)
Improve this answer
2
  • Write-once media is a logistics nightmare, but a simple and effective solution for preventing modification of captured logs (but not access to the logs). Privileged Access Servers are an awesome solution for admin access but does not help if high-priv access is gained to the target through low-priv access (exploits).
    – schroeder ♦
    Sep 23, 2018 at 13:05
  • Wouldn't deploying a mitm server increase too much the attack surface? Is it easyto secure?
    – lalebarde
    Sep 24, 2018 at 13:19
0

One solution is to use a data diode to protect access to the centralized log collector server, which is then only accessible physically, what is easy to control. In addition, specific individual user accounts can be created on it for better protection.

Improve this answer
2
  • Why would a data diode be a solution? What's the problem with 2-way communication with the server if you have authentication processes on the server? A data diode is an over-engineered solution for this use case.
    – schroeder ♦
    Sep 25, 2018 at 15:46
  • Indeed, it is a bit expensive
    – lalebarde
    Sep 25, 2018 at 16:22
0

Filter auditd logs with rsyslog per uid to have a file per uid. Then SELinux can be applyed to implement any policy, especially to prevent one admin to access his own logs. The proposed solution here avoids to require an independant team of admins to access the logs on the log servers when you have a restrictive policy.

As said by others, logs shall be centralized. In addition, access to these centralized log servers shall be performed with local accounts only.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .