Ken Thompson released a paper in 1984 called "Reflections on Trusting Trust", in which he explained that verifiable trust is impossible to achieve.
As you correctly claimed, you can never be sure about these things. You cannot be sure that the program you are using doesn't contain malicious code. You could analyze the source code, but how do you know that the compiler used to compile it doesn't insert malicious code? You could compile the compiler yourself, but how do you know that compiler doesn't contain malicious code?
You could write a simple bootstrap compiler with basic assembler instructions, but you would still need a program to write that to disk and then execute it. How do you know that program doesn't insert malicious code? You could reverse-engineer that program, but how do you know that your debugger or decompiler doesn't include malicious code?
It quickly turns into turtles all the way down. The bottom line is that you can't really verify it without any reasonable doubt. There will always be a "But what if...?" left. But so far, things seem to be working fine, I guess.