If all the computers use the same operating system, attackers only need to focus on one operating system, would it be unsafe?
In 2003, Dan Geer from @Stake published a seminal paper on this very topic - CyberInsecurity: The Cost of Monopoly. Surprisingly (given that he was employed by Microsoft at the time) he comes squarely down into the camp claiming that diversity is vital to security (emphasis mine):
Regardless of the topic – computing versus electric power generation versus air defense – survivability is all about preparing for failure so as to survive it. Survivability, whether as a concept or as a measure, is built on two pillars: replicated provisioning and diversified risk....
...redundancy has little ability to protect against cascade failure; having more computers with the same vulnerabilities cannot help if an attack can reach them all. Protection from cascade failure is instead the province of risk diversification – that is, using more than one kind of computer or device, more than one brand of operating system, which in turns assures that attacks will be limited in their effectiveness. This fundamental principle assures that, like farmers who grow more than one crop, those of us who depend on computers will not see them all fail when the next blight hits. This sort of diversification is widely accepted in almost every sector of society from finance to agriculture to telecommunications.
(He went on to conclude that Microsoft was a threat insofar as it introduced a monoculture; which turned out to be a Career Limiting Move for him at Microsoft.)
In the comments, @Johnny suggests that this is a credentialist answer. While I've chosen to quote a respected professional's well-written paper here, I do so because it mirrors my 20+ years of experience in the computer and security industry. (Which, heck, almost seems like secondary credentialism. But I'm just saying that I'm referencing rather than parroting).
For example, the 3-tier (web/app/db) architecture became a widely accepted improvement in security terms a long time ago, because the separation between differing functions helped enforce security. In that vein, my experience has been that there is a tradeoff between the additional work setting up heterogenous systems (say, IIS with a MySQL-on-Linux backend) and the additional benefit of diversity when attacks (or patches!) introduce disruption into the stack. And that I've regretted having a larger problem more times than I regretted the extra work :)
Virgin field epidemics - either because you're not segmenting your networks, or because you're using the same passwords, or you've only got one [DNS/Hosting/Network] provider, or because you're using the same OS everywhere - all end badly.
If all the systems are the same, then there is predictability of what could go wrong and how to fix and patch. It becomes a lot easier to mitigate problems that you know about. For instance, if the network is all Windows, admins only need to keep aware of Windows risks and deploy mitigations for a single system type.
If you mix the systems, then the admins need to secure that mix to the same level of protection. You introduce a lot of unknowns and a lot of uncertainty, which increases the risks. Yes, it is possible that some systems will be more secure than others, but from a holistic perspective, the more things are the same, the easier, cheaper, and more reliable protections can be.
More diverse systems make it more difficult for an infection to spread.*
Less diverse systems make it more difficult for an infection to start.**
So if your failure scenario is all your systems being infected (e.g. if you need to guarantee uptime), using multiple operating systems can help (if done correctly). If your failure scenario is one of your systems being infected (e.g. if you're a law firm with sensitive documents), using multiple systems will harm.
* A single security flaw that's known to the attacker can potentially allow compromising many or most machines running the same OS.
**A single security flaw, on any of the OSes, that's known to the attacker will be enough to allow compromising one of the machines with that flaw.
This answer ignores the possibility of using different Operating Systems for machines with inherently different attack surfaces (e.g. Unix webserver, Cisco router, Windows workstation), because at that point the answer becomes way more complicated.
As schroeder points out, it would be a lot more work to manage the security of such a network.
But even if managing a diverse network wasn't an issue, humans are still the weakest link of every system. Social engineering is very effective these days, and someone on a Linux box can be tricked into giving up their login credentials just as easily as someone on a Windows box can.
And even if all of your users are savvy about security and never fall for any tricks, the network still wouldn't be practical to actually use. Most of the software that the users will be using to actually do their jobs is likely not available for multiple OSes.
And even if all of your software was cross-platform, you still wouldn't be more secure. The hackers would just start writing cross-platform malware.
It's doable, but it's just not worth it.
Replication and diversification protect against different kinds of attacks.
Replication is the core of any strategy used to protect the business from power outages, fire, water, and other risks where a probability distribution for the occurrence of failures can be given. This knowledge is then used to determine distribution patterns to minimize the risk.
It is an entirely different concern to protect against intentional attacks. Replication can only save a business from intentional attacks if the replicated resources are entirely inaccessible to attackers. Once attackers find out how to compromise one instance of a system, it must be assumed that all identically configured instances can become compromised as well at any moment.
Diversity forces attackers to adapt their strategy to each target. Of course, if the differences are "trivial" to some degree (1), the adaptation might not take much effort. Since the term "trivial" depends on the capabilities of the attacker, putting too much faith into diversity as a safety measure is risky as well.
(1) Debian vs. Ubuntu can be considered to differ too little from each other.
It depends what you are trying to achieve.
Having a diverse set of operating systems in your network will make it harder for an attacker to compromise all computers in your network. But there are very few cases where that's the goal of an attacker.
Instead, attackers usually either try to sabotage your operations by disabling essential services, or try to steal information. In these cases, compromising a single operating system on a single computer might be enough, and having multiple different operating systems in your network means that an attacker is free to (and only has to) find a vulnerability in just one of these operating systems. So here, diversity actually makes the attack a lot easier (e.g. if important files are on a network share that's accessible from Windows, Linux and MacOS desktop computer users, comprising any one of these systems can allow the attacker to gain access to those files).
Like so many things, it comes down to your threat model.
A diverse ecosystem provides an attacker with more than one challenge and the exploit that got him in will not get him on all the other systems.
On the other hand, do you have the knowledge and ressources to keep a diversified network constantly up-to-date and secure?
If you have a bunch of really good windows experts, but no Linux people, then introducing some Linux systems just to get diversity is likely to reduce your overall security as you have no one to properly set up, harden and maintain those systems and they will become a liability.
You also have an easier time automating patch deployment and configuration management in a less diversified environment.
The general opinion, AFAIK, is that if you can run a diversified network, you should do it. But if you lack the expertise or ressources, don't do it just for doing it.
The operating system is at the top layer of the networking OSI model. Layer 7 is the Application Layer. Having a monopoly OS makes exploiting security vulnerabilities more efficient but it also allows patches and resolutions of security vulnerability more efficient.
By making lower level networking protocols and their devices more secure, ie routers, we can compensate for vulnerabilities of OS in application layer. Most intrusions can be prevented with a good firewall router that can handle security at the Session (Layer 5), Transport (Layer 4), and Network (Layer 3) layers of the OSI model before the OS even sees the packets.
As an analogy, imagine if everyone had the same lock--anyone with the key could open every door.
In my job, we have two primary OS. Very many know one operating system but few know the other. The well-versed operating system seems more prone to hacking attempts than the lesser known one. The indication is that the more well known the system, the more likely the need for precaution.
As well, other arenas have shown that a monopoly breeds complacency rather that ingenuity. In the history of Ma Bell, price rather than capability was the driving force. Once it was broke up (albeit a pain), companies broke from the pack with innovation. I think the same is true of the OS of today.
Finally, certain operating systems only function in their markets. OS390/400 works great for fast mainframe "Transactions per second" but not so well for Grandma when she's searching her recipe collection. PC/DOS is great for enthusiasts but can't really do anything beyond nostalgia stirring. Though it is true that some operating systems are mimics of each other (especially in the game console world), each still has it's own playground.
Hope that helps.
Oh, I forgot to mention... a single OS across the business is easier to maintain with patches and so forth but risks all. (A virus could take all the systems out.) Whereas, multiple different OS are more difficult to maintain but also segregates or silos your risks. The adage of "all the eggs in one basket" applies.
In my job, I have to provide multiple levels of redundancy from minor failure to smoking hole resumptions. A good backup is a mirror of the current systems--replicated servers and applications. An even better backup is a mirror of the current systems that function exactly like the current systems but on different operating systems and infrastructure. The best feasible backup is the different operating systems and a third--cloud based--backup. This points to having diversity as a preference over having simplicity.
It's a matter of balancing things to best fit your situation.
There are advantages to 1 for all operating system: you can develop identical security measures, settings, patches and solutions. But the disadvantage is that if one is compromised, attackers can get the rest relatively easy. This is valid for vulnerabilities and not only.
If you have many different OSes, administrating, patching and maintaining overall compatibility may be more difficult, but if one of the systems is compromised, the other OSes will be way more safe compared to the first case.
In both cases, you do need security and patching.
It also matters what OS it is. If it's an easy to be targeted one, it's best if you use others too. If it's one considered safe enough, it's fine if it's only that one.
This is similar to "single point of failure", the failure here is due to homogeneous environment.
But security level actually depends on many factors. Even homogeneous OS environment is not as homogeneous as you imagine, i.e. heterogeneous hardware, OS version, application version, etc.
So the mitigation is all about isolating threat. For example, an organization with many computers will use bridging or VLAN to isolate network according to business operation function.
It will certainly make a hacker's job harder to completely compromise a network once they gain access, if the computers all have different OSes. However, if they have already gained access to the network through one computer, their job has already been made easier. Its doubtful that all OSes will be equally hardened against an attack, so there will always be the weakest link that can be used to attack the entire network.
You also have to consider that compromising the entire network isn't necessarily a hacker's end goal. If all they are after is data, its more likely that the intrusion will go unnoticed, because IT will be spending more resources just maintaining security for all the different OSes.