15

It there a reliable method of “monitoring” system calls under Linux?

There is strace for example to monitor system calls and signals. Is there a way for a process to dodge out of strace? If yes, is there another reliable, secure method of "monitoring" system calls (and, maybe receiving signals), that a process cannot escape from (assuming a proper Linux implementation)?

0

What about monitoring the system calls on the side of the kernel using an external gdb instance.

This could be done by setting up a virtual machine that is configured to run the code of interest. Then QEMU and KVM (by my knowledge) have to be configured to open a port for gdb debugging of the kernel. (See guides blow.)

If this VM is started gdb could be attached to its kernel during boot.

The next step is to set gdb properties and breakpoints to fire on any execve (and consorts) that sets the code of interest as new program. Then let the gdb run until it hits this breakpoint. At this point during the execution of the program the pid of the process running the code of interest could be extracted and breakpoints could be set in gdb that are hit (in the kernel code) on any system call of this process (including fork and execve calls that might lead to additional processes to observe).

In theory this should be a good solution that is hard to doge.

One problem is that everything in the guest system becomes horribly slow, and you might get a huge amount of unwanted calls as bycatch (that you have to filter using gdb...). Additional gdb might have to be extended using python to get the conditional breakpoints working with the required conditions (especially for an automatic child process detection).

Guides how to connect gdb to the guest: Whamcloud Wiki, ReadHat Helpdesk, Stackoverflow

(I did not try these guides. I used gdb some years ago to debug some details of the kernel for a students project. There I used a simple condition on a breakpoint to detect fork calls of a specific process.)

On top of these there are some other techniques to debug a kernel.

PS: Be aware that there are ways to escape a virtual machine (an old example).

  • Welcome on the Security SE! Note, the question was, how to avoid strace, not how to escape a VM. However, your answer is useful - I suggest to find the correct question for it. – peterh says reinstate Monica Aug 4 at 18:44
  • @peterh I wanted to propose "gdb+KVM+Linux Guest with malicious program" as alternative to strace. I try to reformulate my answer to clarify this. – msebas Aug 4 at 20:22
10

If yes, Is there another reliable, secure method of "monitoring" system calls (and, maybe receiving signals), that process can not break (assuming proper Linux implementation) ?

To re-iterate in a slightly different way what D.W. has already said, ptrace is a system call that strace, gdb and the like make to monitor a processes' actions. There are two problems with this approach:

  1. As you probably know, hooking system calls is a favourite technique of rootkit authors. It's entirely possible to replace ptrace, provide you the output of another process or some other such malfeasance.
  2. Processes are not always written to submit willingly to debuggers. You might like to read this challenge set (win32 focused - see the very first entry and keep reading for making it difficult) from an appsec company (I have no links with them). Whilst focused on the IsDebuggerPresent() mechanism, similar solutions exist for ptrace. If you want to see this in the wild and have skype installed on a Linux box, try debugging it.

    Repeating those techniques here, there are two clear anti-ptrace mechanisms:

    if (ptrace(PTRACE_TRACEME, 0, 0, 0) < 0) {
        printf("being ptraced\n");
        exit(1);
    }
    

    This method relies on the fact a process cannot be traced twice. If you can't ptrace yourself, you're being ptraced.

    struct timespec spec;
    
    signal(SIGALRM, SIG_IGN);
    alarm(1);
    spec.tv_sec = 2;
    spec.tv_nsec = 0;
    if (nanosleep(&spec, NULL) < 0) {
        /* EINTR */
        printf("being ptraced\n");
        exit(1);
    }
    

    To explain this one, have a look at nanosleep() and read the original article. In simple terms, nanosleep() is a non-restartable system call and will return early when a signal is handled by the process. This particular process will, when not being debugged, just not handle that particular signal and so will not be woken up. However, a ptraced process will handle it, causing nanosleep to return early. Another example where this happens is the select() system call.

Ultimately, you can mitigate the effects of 1 by ensuring the integrity of your system prior to starting and applying sufficient security measures and an appropriately configured kernel.

What can you do reliably about 2? Not a lot without modifying the original binary code, since any technique for debugging is going to introduce observable inconsistencies or implementation issues somewhere.

tl;dr ptrace will help you if the target process wasn't written with debuggers in mind.

  • You can also check /proc/self/status to see if you are being traced. – forest Oct 18 '18 at 7:20
12

On Linux, you can reliably monitor a selection of system calls or file accesses with the audit subsystem. Make sure the auditd daemon is running, then configure what you want to log with auditctl. Each logged operation is recorded in /var/log/audit/audit.log (on typical configurations).

You'll find simple examples of auditctl usage on this site, on Server Fault, and on Unix Stack Exchange.

strace or associated programs using ptrace are reliable ways of monitoring system calls, but I would be wary of using them on a malicious program. I couldn't tell you how off the top of my head, but it should be possible for a monitored program to make the right ptrace calls to evade monitoring.

Note that a malicious program could spawn a process that is not audited and can execute code that won't be logged. For example, it could use mmap to write to a file without the file contents ever appearing as the arguments of system calls, make this file executable and spawn a process executing it. The spawned process can typically break the process tree with something like ssh localhost. If you audit all the processes executed by a given user (as opposed to only a single process and its descendants), you'll be able to log everything.

  • Beware, however, that (to my knowledge) auditctl does not provide a way to record all system calls made by the process and their parameters. As a result, it may or may not suffice for Grzegorz Wierzowiecki's needs. – D.W. Oct 30 '11 at 17:49
  • 2
    @D.W. What issue do you see with auditctl -S all -a exit,always -F uid=1234? As far as I know, it does record all system calls, but it doesn't show the data passed through pointers. – Gilles 'SO- stop being evil' Oct 30 '11 at 19:17
  • 1
    You've explained the limitation well: it doesn't capture data passed to a syscall through pointers. For instance, strings are not recorded. That's a significant omission. A lot of security-relevant information is passed through pointers, and thus will not be recorded or visible in the audit output. – D.W. Oct 30 '11 at 22:53
  • Do I get it right : auditctl records all syscall but doesn't process parameters in all cases ? In my case actually it's enough. On one hand I can do some stuff with just killing when not allowed syscall is done, on the other hand I'd like to make same "wrappers" (but I've wrote about it in other topic). Thanks for pointing me that missing feature. – Grzegorz Wierzowiecki Nov 11 '11 at 22:13
  • 2
    @GrzegorzWierzowiecki Yes, The audit subsystem is at its most useful when monitoring file accesses, as it shows what files were accessed in what manner by which process. For other types of syscalls, you'll see a lot of pointers, which don't help if you're after the data that they're pointing to. strace is king if you're after the data, but even it can't show everything (e.g. data written into an mmap'ed file). – Gilles 'SO- stop being evil' Nov 11 '11 at 22:26
4

Yes. strace is a reasonable way to monitor system calls and their arguments, as long as the process being monitored is not malicious. If the process being monitored is malicious and was written to evade strace, I expect that it can do so. strace was not written as a security tool, and I can hypothesize several ways that the process might defeat it. See, e.g., Robert Watson, Exploiting Concurrency Vulnerabilities in System Call Wrappers or Tal Garfinkel, Traps and Pitfalls: Practical Problems in System Call Interposition Based Security Tools.

If you are worried about malicious code, you'll want to use a sandbox that was designed for security, rather than a tool like strace that wasn't designed for security. The general approach to building such a sandbox is to use system call interposition both to contain the monitored process, and to monitor its actions. One portable method is to use ptrace, though this can introduce a non-trivial performance overhead as it forces a context switch on every system call. On Solaris, you can use /proc; /proc lets you specify the subset of system calls that you are interested in wrapping, which lets you achieve better performance at the cost of compatibility.

Take a look at Plash, Systrace, and Subterfugue, to see some worked systems that use these sorts of methods. Also look at Chrome's sandbox, which uses a variety of mechanisms (including seccomp on Linux).

  • Thanks for all tips. In mean time I'll try to dig into all referred stuff. As you pointed out correctly, I'd like to run malicious code, so it should be more sandbox like, thank monitoring design. Because I'd like to add such functionality as part of whole system (preferably code in) small tools, of C/C++ code is preferable. Could you refer to any more useful materials related with all this stuff ? (article, book, sourcecode). I'd love to check your current references, soon. :) – Grzegorz Wierzowiecki Nov 1 '11 at 14:34
  • Btw. I know about about Chrome's snadbox for longer time, but as I remember it from last year, it's designed as part of whole, not to use standalone. seccomp is one of mechanisms I'd love to use, but for it I need at least to wrap "brk" system call. (I thought about doing it with binary instrumentation with valgrind, but again - it's not trivial to me ;), so I look in parallel for other solutions ). – Grzegorz Wierzowiecki Nov 1 '11 at 14:38
5

The Linux Audit Framework supports syscall monitoring - I believe it is what you are looking for.

  • See my comments on @Gilles's more detailed answer for an explanation of why this might or might not meet the needs of the original poster. – D.W. Oct 30 '11 at 22:56
  • Sorry but, indeed , Linux Audit Framework does not suit my taste. I'd like to code such functionality inside my program, so existing app, can help me by stuidying it's sources. LAF looks big, so I assume, that studying it's sources might be not best source of information for me. Even thou - thanks :). – Grzegorz Wierzowiecki Nov 1 '11 at 14:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.