I'm learning about buffer overflows and that there are various different types of them, including ones you can do when you can't directly hijack the return address.

In these cases, you can hijack a function instead and make that call a shell for you. It took me days but now I finally think I understand it. But much to my surprise the payload doesn't work.

This is essentially the function call I'm trying to hijack:

  fprintf(stdout, "%s\n", buffer);

So I overflow the buffer, rewrite a pointer to point to [email protected], and write the address of system() there.

I also use the string "/bin/sh;#" repeating to fill the buffer, expecting it to be passed as the argument to system(), but instead what I get is:

sh: 1: ▒: not found

Any idea what's happening here that I'm not seeing?

As @Meow pointed out below, it seems like whats happening is that when the fprintf call gets replaced, the command becomes:

system(stdout, "%s\n", buffer);

which is effectively just system(stdout). So system is trying to run stdout instead of my argument which is in the buffer at this point. Spent some more hours trying to figure it out, tried to do some stuff with IO redirection when running the program, but nothing I was trying worked.

Is this even possible?

EDIT: It seems this is actually not possible to do like this. If you're going to replace a function with system(), the arguments to that function have to actually be something you can use with system()

  • Can I see the result of checksec? Has RELRO been shut down?
    – niyang dan
    Nov 6, 2022 at 10:59
  • @niyingdan I believe the RELRO on it is set to partial - I can actually see & confirm the GOT entries are being hijacked, but it's on an isolated system where checksec isn't installed and I don't have perms to install it. A small caveat is that I can't point directly to the [email protected] entry, because there is a \x20 character in the address which gets stripped from my input to \x00. I was able to get around this by pointing to the byte before and writing the address of system twice. example Nov 6, 2022 at 16:46

1 Answer 1


The first and only argument to system() is a string containing your command. Because you're overwriting fprintf's address in the GOT with system, system ends up taking stdout as its argument, not the shell command you're expecting.

When hijacking control flow by overwriting addresses in the GOT, you need to make sure the function arguments are as you expect. One way to do that is to use your debugger to break on the hijacked function and check that the arguments look as they would if you were writing a benign program using that function.

  • I see, thanks. I was thinking along the lines of something similar, but I'm having a real tough time figuring out to get past that in this case. This is my stack, and I see clearly where the shell command I put into the buffer is located, but I'm not exactly sure which one is stdout and if there's something special I need to do to replace it. I tried to pump some more characters into the buffer, and that overwrote the lower part of the stack on subsequent runs , but that still never ran my command. Nov 6, 2022 at 19:38
  • 1
    @A.Trevelyan if you control the instruction pointer and some of the argument values on the stack, my suggestion would be to use a different function to hijack control flow. If this is a challenge program you wrote yourself, replacing fprintf with just printf would be enough to demo this exploitation technique. Otherwise, I recommend looking for another place to redirect control flow to (and perhaps look into writing a ROP based payload instead.)
    – Meow
    Nov 6, 2022 at 23:31
  • It is a challenge program, but I didn't write it myself. This is the full source code for it, I don't see another function to hijack really. As far as I can understand, you use the 1st strcpy to overwrite p, the 2nd to overwrite somewhere else, and then the only things left are fprintf or exit. I tried messing around with exit, but nothing I was doing there was working. Nov 6, 2022 at 23:43
  • You could make the exit benign (f.ex overwrite with printf), thus allowing the traditional stack overflow to work, but you're well into rube goldberg territory at that point, which may be the intention if its a harder CTF challenge
    – wireghoul
    Nov 7, 2022 at 22:14
  • @wireghoul Yeah, apparently the solution was to just not bother with system() at all -- put a shellcode into the buffer and then rewrite fprintf with the buffer address instead. Not sure why I couldn't come up with it on my own but I was also kind of deliberately misdirected at the start. Nov 8, 2022 at 4:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .