Sometimes Linux users need to to download a source code to be compiled then executed (the root privileges is granted).
Is it possible for a source code to hide malicious code as a part of it? And how does one verify that the source code is safe before compiling?
Short of reading and understanding every line of code and how it all fits together, you realistically can't. The best you can really do is to download it as a package from a reputable source who vet the packages in advance, to minimise the risk for the user.
However, there are times when even full blown operating systems are hacked (such as Mint Linux earlier this year) so you can't really be sure what you're getting is safe.
As with most things I've found when thinking about security, a lot of it comes down to balancing trust and common sense.
No. It is not possible to verify code.
The mathematical analogue to this question is called the "program Verification" problem. Program verification is known to be impossible to decide. This means there is no algorithm that can check your computer code for correctness. Closely related, the halting problem is not decidable, meaning there is no algorithm that can tell you if your code will ever stop running. This was all proven back in the 1950-1960s, and everyone who gets a bachelor's degree in computer science should be taught about these results. It is mathematically impossible to verify code.
Even when reading through the code, it is not possible to verify it. In some cases, very short codes or algorithms can be proven correct using mathematical logic. Proofs only work for core algorithms, such as the sorting algorithms and graph traversal algorithms. For any large sized code-base, such as an operating system, there is no way to prove correctness.
Some static analysis and some formal methods tools exist, and all these tools attempt to do some limited form of verification. These tools try to prove what can be easily proven about the code. Static analysis tries to define provable properties of the dynamic behavior of the code when it executes. Some systems use asserts while others use loop invariants. However, in all cases, static analysis cannot perform full program verification.
If you decide to run a code you are implicitly trusting the authors of the code. You are trusting their intent to write code that works and that respects typical security policies, such as access authorization, permissions, etc. Tools such as mdsums can help you decide if you downloaded the version of the code that the authors made available, but they cannot help you verify the design of the code.
Yes, if one blindly downloads and compiles source code, that code could contain an exploit that, if run, could harm one's system. What's more, the resulting binary may not need to be explicitly run. In his 1984 classic Reflections On Trusting Trust, Ken Thompson demonstrated how one might go about creating C code that, when compiled, exploited the compiler and backdoored the system upon which that compiler resides.
There are some ways to defend against this. In 2006, Bruce Schneier blogged a pretty good breakdown of a paper by David A. Wheeler on defending against Thompson's specific example. The paper itself is still paywalled as of this time, to the best of my knowledge.
The Wheeler paper is very interesting, but it is focused on the bowels of compiler design, and this question seems more focused on end-user precautions than compiler design or even systems programming. There are generally two ways we understand the risk involved with compiling a specific piece of code:
The second case--a thorough code audit--is a huge, long, resource-intensive task. It almost never really happens for codebases of nontrivial size, because it is simply too costly. Much more often, we are looking at the first case: trusting the coder, and validating that the code hasn't been tampered with between the coder and the consumer.
In 2015, I did an article for LinuxJournal on how code typically gets from the developer to the Linux user. Chain of Custody has since been republished by my employer outside the LinuxJournal paywall. It's heavily focused on the path that goes through package management, but if you read carefully, you'll realize that the parts applicable to a package maintainer compiling code obtained somewhere on the internet are also applicable to an end user doing so.
Of course, in the end, as others have pointed out, these integrity checks only do one any good if the developers' infrastructure hasn't been compromised, if the developer was coding well, and so on.
I think what you are asking for is "static analysis" (as opposed to "dynamic analysis" which analyzes running code).
The web security group OWASP has a page talking about static source code security analysis in broad terms. If you simply google for "static source code security analysis" you will find many product available (some payware, some free).
The part of the answer that you're not going to like is: typically these analyzers are looking for vulnerabilities in the form of buffer overflows, use-after-free, and stuff like that. It's almost impossible for a program to decide what counts as "malicious behaviour", for example the program may read all your files and upload them to a server. This is perfectly fine if the program is a cloud sync tool like Dropbox, but less fine if it's a solitaire game. Bottom line: there's no substitute for reading through the source code yourself.
Personally, I look at the reputation of the site I got the source code from. If it's on github or similar, who are the main contributors? If I google them, do they seem legitimate? The code itself was downloaded by HTTPS or SSH, right? I might look up some reviews of this software to make sure no bloggers have raised red flags about it, etc. (Final note: the risks of stuff like this is much lower if you're installing the software as a package through your distro's package manager, so always prefer this if possible.)
As others have mentioned, actually verifying that code has no malicious behavior is neigh impossible. There are a handful of special cases, where code was written to be verifiable, such as the seL4 kernel, but the general case is easily demonstrated as unsolvable.
In practical situations there are steps you can take to decrease the likelihood of an issue:
