6

Two-man control can be a strong defence, but the overhead is a concern, as discussed previously.

How can two-man control be implemented with minimal overhead?

I would be most interested in a concrete example: an online banking platform. The environment includes a load balancer, web servers, database servers, and a link to the core banking platform (which is out of scope).

If the two men collude then they will be able to perform malicious actions on the system. That is an inherent limitation with two-man control, and we should not attempt to solve it.

To avoid this question being too broad, lets only consider sys-admins - not developers, support agents, etc.

  • 1
    I agree the question fits here. I would suggest you add clarification to the scope. When you say "sys-admins", can you provide example tasks that they would perform, which you see in-scope for the two-man rule? Are we talking root server access, router login? Example tasks would be helpful. – Bryan Field Jul 29 '16 at 17:22
  • 1
    @GeorgeBailey - Sys-admins will have root/admin access on all systems, and unconstrained root access will certainly need two-man control. Examples of sys-admin tasks? Deploying application updates, patching software, troubleshooting. You know, sys-admin stuff :) – paj28 Jul 29 '16 at 19:49
  • 1
    Don't get cheap on me, that was Hammond's mistake! -- Jurrasic Park's crazed computer programmer – Mark Stewart Jul 30 '16 at 17:57
  • Since the question came up when talking about the overhead associated with it, I want to throw just one thing in: The problem isn't the technical side (although there are challenges there as well). The problem is that another person has to carefully review every single line of every script person A wants to execute. This might sound easy on first glance, but think about it: There are so many complex interactions in modern systems that it's really hard to figure out all possible security vulnerabilities in a script and that's without someone intentionally trying to smuggle something in. – Voo Jul 31 '16 at 15:54
  • 1
    The bad guys bribed the crazed computer programmer to smuggle out some dinosaur embryos. The bad guy met with him at a cafe to finalize things, and the bad guy hesitated to pick up the check. Then the quote: imdb.com/title/tt0107290/quotes?item=qt0463093 – Mark Stewart Jul 31 '16 at 21:41
2

There are a lot of good answers here. In the end it is a personal decision and the organizational process is also a very important part of the whole process.

I would like to point to another, better managable solution. Disclaimer: This is a solution I am developing.

privacyIDEA is a two factor management system. Or to put it more broadly it is an authentication management system. One possibiliy is, you can also assign an abstract authentication token "4-eyes-token" to a user account. This way you can setup that a certain user account or certain commands (sudo) need authentication by two or more persons. These users could authenticate with passwords or with one time password hardware tokens like yubikeys.

The nice side effect is, that you get a digitially signed audit log within privacyIDEA, so that you can know, which two persons acted within your two-man-rule.

  • Nice project - congrats on the Thomas Krenn award! This is the first software I've seen that explicitly support two-man control, thanks for pointing me in the right direction :) – paj28 Jul 30 '16 at 16:31
2

How can two-man control be implemented with minimal overhead?

TL;DR This can be accomplished using a special sudo command that requires a second person to approve each command, combined with a nice whitelist to make safer commands more convenient. It is important that each command is approved by the two-man rule or else the protection is easily defeated.


Two-man rule implementation options:

Unconstrained root access

Enable root access temporarily

A small root script could require a second person (User A) to authorize before sudo becomes available for the acting sysadmin (User B)

  1. User A: enable-sudo user-b
    (auto-adds an entry to /etc/sudoers with either a time-limit or scheduled removal via at job)

  2. User B: sudo do stuff will start working at this point

Unfortunately, a back-door could easily be added by User B once User A has authorized the access. This could be as simple as re-adding user-b to /etc/sudoers or starting a separate service.

It is not possible to robustly enforce security rules once someone has gained root (as it is "unconstrained"), so you should covertly monitor the most likely bypass possibilities and alert another (more trusted) sysadmin if something begins looks odd after a two-man enabling of sudo.

  • unexpected at or cron job
  • unexpected background process still running
  • unexpected startup script

And don't tell them exactly what safeguards you are using. This will help against the obvious back-doors, especially if user-b is not familiar with the protections in place, but will not thwart a user-b who is determined to cause trouble.

Command-basis two-man rule

Create your own version of sudo, which alerts the second person of any command suggested by the first person, with the ability to approve or deny.

I won't go into the implementation details here, but it would be fairly straight-forward to implement.

In this case, a back-door would have to be hidden within one of the provided files, for example a software update to be installed.

File editing with vim

Your custom version of sudo should not allow launch of direct file editing commands. If someone types oursudo vi, then oursudo will have to launch a special script for file editing.

  1. Copy the file to a /tmp location (randomly generated filename to prevent symlink attacks)
  2. Launch vim with reduced privileges to a non root account (so user-b cannot use the "Save As" function within vim to bypass this wrapper)
  3. When saving is completed, present a diff of the file changes to user-a.
  4. Once approved, copy the approved file from /tmp to the target location.

Physical barriers

Requiring physical access to gain root access could help bring visibility to what user-b is doing. Depending on the office environment, the team may have a clue to what user-b is up to, and/or user-b may be less comfortable causing trouble in that environment. It depends on how often such access is necessary though.

Shortcomings: root = unconstrained

As you can gather from the ideas I've brought up, there does not seem to be a sure-fire to protect against back-door installations. root access is designed not to allow this.

Whitelist commands approach (constrain the root access)

The best thing is to create a whitelist of available commands. (which sudo has the ability to offer) This does create a significant overhead at first, but eventually may become more practical as you learn the command that are commonly used.

Deploying application updates

Your applications should not run on root access. Those that do must go through a separated Code Review system and then updates from the approve VCS commit can be included.

Patching [packaged] software

This is cannot be constrained, so I would suggest using command-basis two-man rule outlined above.

Troubleshooting

Safe tools can be permitted via sudo, and then use command-basis two-man rule to roll the fix.

  • 1
    Re root being unconstrained: For what it's worth, though very hard to configure, SELinux (especially RBAC/SELinux users) would be able to severely limit what a user as root could do. A policy could be written that would prevent a user from changing their SELinux user and even as a Linux root, they could be very effectively contained. – Naftuli Kay Jul 29 '16 at 23:22
  • Nice answer! Granting user-b root access - even just temporarily - violates the two man rule. I like your idea of custom sudo, although how would they edit files? If user-a allows "sudo vi xxx" then user-b can get unconstrained root (:! within vi). The app won't run as root, but access to the app account must be subject to the two-man rule as well. – paj28 Jul 30 '16 at 16:29
  • @paj28, Great question. I've edited in the potential file editing solution. – Bryan Field Aug 1 '16 at 12:22
1

The key is to only implement two-man control for operations that really need it.

Routine tasks

Routine tasks, like deploying an application release, or applying security patches can be scripted. One sys-admin can create the script, but it cannot be run until a second sys-admin has reviewed and approved it. Common tasks like restarting the application server, or deploying the latest application release, can have a pre-approved script that is used repeatedly. Non-standard tasks will need a bespoke script.

To do this, there's a need for some kind of workflow management system. You may be able to use a ticketing system like Trac, build a bespoke system on something like JIRA, and configuration management tools like Puppet may have features that would help. The two sys-admins do NOT need to be physically local; they can work remotely.

The systems should be designed so that as much routine maintenance is automated as possible - scheduled log rotation, etc.

Diagnostics

Sys-admins can have read-only access to logs without two-man control, provided the logs mask sensitive information. They can also have a low-privilege account to view status pages, and run commands like top or ps. This allows a single sys-admin to diagnose issues - and they can work remotely. They may be able to do remediation by submitting a script as a routine task. However, in some cases that won't be possible, so there is a need for emergency access.

Emergency access

Sometimes there is a serious issue and you just need to SSH in as root. To maintain two-man control you need two people at this point. In practice, they need to be physically next to each other. A simple way to implement access control is to have an account for each pairing, where each sys-admin knows half the password. You could create a smarter system with a custom PAM module, perhaps even requiring a separate smart card from both sys-admins.

Once logged in, it is down to the sys-admins to maintain two-man control. They must inspect each others commands before issuing, and lock the terminal if one of them needs to step out.

The aim should be that emergency access is rarely needed.

Development environments

It is only necessary to use two-man control for the live environment. A lone sys-admin can have full access to a development environment - provided that live data is not present. They can try out a task in the development environment, perform some testing, iron out the bugs. Only when they have got a working script do they submit it to another sys-admin for approval.

Conclusion

If all this is done, I believe you can implement strong two-man control with about 25% overhead. Making it a success relies on various things (like automated maintenance) that are good practice anyway. If a system is so sensitive as to need two-man control, it is a good idea to get all the basics right as well!

I have never seen this implemented in a commercial or government environment. The closest I have seen was an online banking platform where most sys-admins did not routinely have root access. To performs a task, this would first be planned in detail, then approved on the change control system. To implement the change the sys-admin then got root access for a limited period. Also, there were a small number of senior sys-admins who had permanent root access, in case of emergencies. This arrangement is much better than I typically see - although it falls short of full two-man control.

1

There's a cryptographic technique that is very relevant here, called secret sharing:

Secret sharing (also called secret splitting) refers to methods for distributing a secret amongst a group of participants, each of whom is allocated a share of the secret. The secret can be reconstructed only when a sufficient number, of possibly different types, of shares are combined together; individual shares are of no use on their own.

So one strategy that you could adopt is:

  • All administrative actions require authentication with a cryptographic key.
  • The key is split n ways across your n administrators, with a two-share threshold to recover the secret.
  • Every administrator must keep their share secret from all the others.

I don't know of many implementations of secret sharing out there. See this question for references. I would add this:

  • HashiCorp's Vault is a client/server distributed system for storing and retrieving secrets (e.g., passwords that must be accessed by applicaqtions). The contents of the Vault database are encrypted with a key that's not stored anywhere, but rather split into a set of secret shares with configurable number and threshold. To reconstitute the key they have a process they call unsealing where the owners of the requisite number of shares must each enter their share.

So it might be possible to cook something up with Vault to achieve this. The key ideas would be:

  • Keep all the admin credentials inside a Vault instance that's only unsealed when they are needed, and sealed right away when they're not;
  • Script all your tasks so that admins never see the actual credentials, only their Vault master key shares.
  • Thanks... I'm sure secret sharing would have a part in some two-man systems. In the case of administering a server though, it's probably better for the server to require two logins (like privacyIDEA). I'd be concerned that admins could get a peek at the admin creds when they unlock the vault - and then later use that for lone-man access. – paj28 Jul 30 '16 at 16:33
0

The only way two-man control can be implemented efficiently without any possibility to bypass it, is to physically secure the resource in question (computer, data media etc) so its inaccessible unless 2 persons, via biometrics, authenticates to the service.

I would NOT recommend a 2 man rule for login as root or similiar for a computer, instead, I would rather recommend restricting so that type of access is only available on-site on a physical terminal, and then that physical terminal is simply tucked into a locked room (server room or whatever) requiring 2 man to authenticate to unlock.

The reason, is that otherwise, either one man can maliciousy plant a backdoor to allow single man control, or the two mans can collaborate to reduce system security to single man control.

By physically restricting to two-man control, its not possible to bypass the system, even if the two mans collaborate to give access to the single individual because the other man thinks "he is trusted" or something like that.

(of course, two mans could collaborate on executing a malicious command or whatever, but the important factor here is that they cannot collaborate on giving permanent single man access)

  • If the two men collude, then all bets are off. They could configure the system to allow remote access from outside the secure area, and one of them could get single-man access in future. Otherwise... nice answer :) – paj28 Jul 29 '16 at 20:18
  • @paj28 Thats why the area and secure room must be made so no compromise can be done. Can do one example on how it can be implemented: Imagine a drive whose rw/ro-switch is controlled from badge reader. This drive contains a web server and associated web page. Anytime a change needs to be made, 2 man needs to visit said physical room, and make the changes. The 2 man can collude to make malicious changes to the website, but not collude to allow single man access in the future (without doing really bad things like physically opening things that could be catched by security camera for example). – sebastian nielsen Jul 30 '16 at 0:23
  • In that case they could create a CGI script that executed commands, then go home and control the server from home. Anyway, I'm going to edit the question to make clear that we're not attempting to solve the collusion problem. – paj28 Jul 30 '16 at 16:33
  • @paj28 Because when they are at home, the rw/ro switch is in read-only mode (controlled by access control system) so no (permanent) changes can be done. If the collusion problem with reducing security, is not present, eg the problem with "I trust you to do things right, so here is my part of the access, use it responsibility" does not exist, then a two-man control can be very simple as dividing the root password in 2 pieces and giving one piece to each. Thus the question does not have any relevance. – sebastian nielsen Jul 30 '16 at 23:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.