I'm writing an assignment for a security course, and I'm trying to create an executable which students can interact with (ie, execute), but not inspect. In particular what I'd like is for them to be able to execute the binary but not be able to inspect its contents in any way - reading the file, attaching a debugger, etc.

From trial and error, it seems as though giving "other" only execute permission, non-owners of the file are able to execute but not read or attach a debugger (including by attaching to it after it's running - it's actually ptrace that gives the error, so I'm convinced that it's not just gdb failing to read the associated binary file). My question is: am I right about this? Is giving students only execute permissions actually sufficient? To be concrete, they're asked to crack encryption, and the key is stored in the binary, so if they can inspect the binary in any capacity (including the process' memory), they'll be able to sidestep the challenge of the assignment.

  • 2
    Off Topic: Not a real life scenario in my opinion. In addition to that, it's a bad practice to store the encryption key in the executable. I'm not seeing the actual point is such an assignment, perhaps you can elaborate on that.
    – Jeroen
    Nov 11 '15 at 5:18
  • if your students can extract the key from the executable and use it to decrypt the cyphertext ..... win?
    – schroeder
    Nov 11 '15 at 5:33
  • @Jeroen-ITNerdbox for me this is a fairly common scenario... but then again I'm in teaching too :-) Nov 11 '15 at 12:09
  • 1
    Suggestion: make a web application that performs the same function. Handle the secret parts on the back end so that (assuming security) the process cannot be inspected at all. This would solve most of your problems. Nov 11 '15 at 15:59
  • @korockinout13: If I wanted to go that route, I'd just setuid the binaries (which I know is a viable option - just one I'd rather avoid).
    – joshlf
    Nov 11 '15 at 18:14

This is a partial answer as I've got little time...

Your approach would prevent plain reading of the file but students could still cheat, e.g. by LD_PRELOADing, and changing any of your internal functions to literally print the stack/heap on screen. You must prevent changes to the environment as well (you can unset LD_PRELOAD as a first thing in your main()).

Students could also strace your program, and that might or might not reveal internal strings depending on the system calls you make with them. However, according to Gilles' comment on this question, strace will not honour suid/sgid bits when executing.

To prevent both gdb/ptrace attaching and strace, you can make the program suid/sgid to a specific identity that has access to a file chmoded to 400. This will mean that the executable must be run with that suid identity to access the secret. In fact, if I remember well, LD_PRELOAD should also be ignored by suid binaries.

You can easily verify this by building a program that prints the results of geteuid() and getuid() and run it, chowned to root and with the setuid bit, both normally and with strace.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.