1

If C uses shared libraries, is it possible to write a program that will modify the fopen function to execute a malicious code? How can the OS protect the user from this?

1

Yes, that is possible. This technique is called function hooking. In fact, you can completely control an executable by hooking, say, __libc_start_main and redirecting execution to code in the library. There is absolutely no way an OS can protect an executable from a malicious library. The code in the library will run in the same address space as the executable and will have the exact same abilities. The only way to protect an executable from a malicious library is to not link it with malicious libraries.

  • The interesting about that is: antivirus can't get that from my understand, right? – Faminha102 Oct 30 '18 at 2:00
  • @Faminha102 Antivirus can't get what? If you start antivirus from a compromised environment, then you could load a malicious library with the AV, but generally it is started from a privileged location (e.g. as a service), which means that you'd need to already have high privileges to provide it with a malicious library. – forest Oct 30 '18 at 2:15
  • 1
    @forest I assume OP meant that an Antivirus can't detect that you're trying to make some innocent program load a malicious library and thus execute malicious code. Probably OP is trying to put it into action. – Sombrero Chicken Oct 30 '18 at 10:27
  • @SombreroChicken Ah. Well AV will definitely try to scan the library, but obviously you could avoid AV detection in libraries just as you can in executables. – forest Oct 31 '18 at 1:49
0

Also, to be clear: when you modify a shared library in memory, only the copy of it in your process' address space gets modified. Neither the copy on the disk nor the copies in use by other processes will get modified. It's the same as modifying a global variable in a process (such as errno); only the one process using that address space will "see" the modification.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.