17

In Java, normally permission checks are handled by the SecurityManager. To prevent untrusted code from invoking privileged code and exploiting some bug in the privileged code, SecurityManager checks the entire call stack; if any of the callers in the stack trace are unprivileged, by default the request is denied. At least, that's how the standard SecurityManager checks work.

However, a few special Java APIs follow different rules. They bypass the standard SecurityManager checks, and substitute a weaker check. In particular, they check only the immediate caller, not the entire call stack. (See Guideline 9-8 of the Java Secure Coding Guidelines for details. The special APIs include, for example, Class.forName(), Class.getMethod(), and more.)

Why? Why do these special APIs bypass the standard checks and substitute a weaker check? And, why is this safe? In other words, why is it sufficient for them to check only the immediate caller? Doesn't this re-introduce all of the risks that the standard SecurityManager checks were designed to defend against?

I first learned of this when reading an analysis of the recent Java zero-day exploit (CVE-2012-4681). That analysis deconstructs how the exploit works. Among other things, the attack involves taking advantage of the weaker checking done by these special APIs. In particular, the malicious Java code manages to get a reference to a trusted system class (through a separate bug), then it fools that trusted system class into invoking one of these special APIs. The resulting permission check looks only at its immediate caller, sees that the immediate caller is trusted, and allows the operation -- even though the operation was originally initiated by untrusted code. Thus the weaker checks don't stop the attack, but as far as I can see, this attack would have been prevented by using the standard SecurityManager checks (since the caller's caller is untrusted). In other words, this recent attack looks like an example of why the weaker check is risky.

However, I know the Java designers are smart folks. I suspect the Java designers must have considered these issues and had some good reason to bypass the standard checks and substitute the weaker checks for these special APIs -- or, at least, thought they had a good reason why this was safe. So, maybe I'm missing something.

Can anyone shed any light on this? Did the Java designers screw up with these special APIs, or were there valid reasons for substituting the weaker checks?

Edit 9/1: I'm not asking about how the exploit works; I think I understand how the exploit works. I'm also not asking why, in this particular example, the trusted code that invoked these special APIs was buggy. Rather, I'm asking why the special APIs -- like Class.forName(), Class.getMethod(), and so on -- are specified and implemented to use the non-standard weaker permission check (only look at the immediate caller) instead of the standard SecurityManager permission check (look at the entire call stack). This design decision (of using weaker permission checks for those special APIs) allowed the recent vulnerability, so it would be easy to criticize the design decision. However, I imagine there might have been some good reasons for doing things this way, and I'm wondering what those might be.

0

2 Answers 2

9

(This is only a general comment on the "why", not on the specific attack you are alluding to.)

Unfortunately, the Java designers found that it was highly possible to paint yourself into a corner, structurally speaking. For instance, there are classes in java.io and in java.net, which are both involved in doing I/O. Let's assume that a given JVM has special OS-interaction native code in java.io.FileDescriptor, which allows it to do send() and receive() system calls (that's not the case of the Sun/Oracle JVM, but it could happen in another JVM, and indeed it does in at least the one I once wrote). To enforce the sandbox semantics, these methods are not public, of course.

The implementation of java.net.Socket would like, quite naturally, to use these methods. But, as per the naming conventions, Socket is a class which is located in the java.net package, not java.io. How can it access the non-public methods of java.io.FileDescriptor, considering that it must do so even if invoked from non-trusted code (non-trusted code can open sockets, albeit not to every destination) ? There are mainly two ways:

  • Add some "bridge" native methods in java.net.Socket, which forward the calls to the methods in java.io.FileDescriptor. Native code sneers at packages and visibility; native code can do everything.

  • Allow the code from java.net to do some reflection to access non-public methods in other packages. Code which can do that can do about everything, because it could modify the data structures used by SecurityManager itself (hey, I remember that you pointed that to me yourself, maybe ten years ago, in a Usenet discussion where we were both using our real names). So, unbounded reflection must not be granted to everybody, but, in the situation I describe here, it must be granted to code in a specific system package (java.net) even though that code is called from an untrusted applet.

The second method requires the weakening of the security model. Actually, it is quite generic: if the untrusted applet must do anything useful at some point, it must be able to access the system (if only to display the result of computations or send it back to the server), which needs a kind of gate. Native code is such a gate. The security weakening model is another kind of gate, which has the added benefit of staying in the "pure Java" world (I can understand that in the JVM maintenance team, native code is probably frowned upon because it makes things more expensive). The bad side is that by weakening the security model, the attack surface is greatly enlarged: now, all the Java code in the packages-with-privileges has become critical. To reuse @Hendrik's analogy, it is about giving a root setuid bit to a substantial chunk of code (from the java.* packages).

In particular, the way that Java weakened the security model is to designate some special APIs -- like Class.findClass(), Class.newInstance(), Class.getMethod(), and other reflection-related APIs -- as using weaker permission checks. In the example above, this allows trusted system code in java.net.Socket to use these special APIs to get a reference to a non-public method in java.io.FileDescriptor and invoke it reflectively.

  • For instance, the java.net.Socket code can use Class.getMethod() to get a reference to the non-public method in java.io.FileDescriptor (this is allowed since the immediate caller of Class.getMethod() is java.net.Socket(), which is trusted code) and then invoke it.

    Notice that this implementation strategy relies upon Class.getMethod() to use the weaker permission checks. It just wouldn't work, if Class.getMethod() used the standard permission checks. If untrusted code invokes java.net.Socket, and then the java.net.Socket code calls Class.getMethod(), the standard permission checks would reject this call because there is untrusted code somewhere on the call stack, and the java.net.Socket stuff wouldn't work right (in non-attack scenarios). In contrast, the weaker permission checks used in the weakened security model do allow it.

So, the weaker permission checks help the Java designers get themselves out of the corner they painted themselves into.

Supposedly, with a "perfect" structure of the system code and naming conventions, a handful of native methods would be sufficient, but the presence of specific is-immediate-caller-from-a-trusted-package calls in the JVM system library code shows that the structure of said code is not perfect.

4
  • Wow! Great answer! Thank you for vividly and clearly explaining the reasons behind this design decision.
    – D.W.
    Sep 1, 2012 at 18:34
  • I edited your answer to add some more explanation of how specifically the permission model is weakened and how this enables Java designers to get themselves out of the corner they painted themselves into, with concrete code examples. Hope this looks good to you, but if not, please feel free to roll back my edit.
    – D.W.
    Sep 1, 2012 at 18:45
  • 1
    Follow-up question: isn't this the problem doPrivileged is designed to solve? An alternative design would have been to use the standard permission model and standard permission checks for these special reflective APIs, and have the java.net.Socket code call doPrivileged() (or enablePrivilege()) before invoking the special API (e.g., Class.getMethod()). That would also allow the Java implementors to get themselves out of the corner they painted themselves into, without introducing a weaker permission model. Does that sound right? Do you have any guesses why they didn't do that?
    – D.W.
    Sep 1, 2012 at 18:55
  • 1
    @D.W.: I suppose it is a consequence of the "root of all evil" (aka "premature optimization"). Semantics for access control on reflection are complex: unprivileged code can access private members but only on classes which are part of the same ClassLoader as the caller's. This can be done with doPrivileged() framework but requires creating the appropriate permission objects for each reflective method call, which is probably expensive -- hence the shortcut (which is documented, and hence must remain due to evil #2: backward compatibility). That's my best bet right now.
    – Tom Leek
    Sep 2, 2012 at 14:41
3

Summary

Some trusted methods need more permissions internally to fulfill their tasks. But if those methods have bugs, they may allow an untrusted attacker to execute undesired actions at an increased privilege level.

Background

We have a very similar situation at the operating system level. On Unix there are setuid/setgid flags and the sudo command. They allow an unprivileged user to execute tasks, which require a higher level of privileges internally.

For example a normal user is not allowed to modify /etc/shadow. But we want users to be able to change their password. Therefore the passwd command is flagged as trusted (setuid root) and allowed to modify that file. Of course it has to perform its own security checks.

The same situation applies to the Java sandbox. For example unprivileged code is not allowed to dynamically invoke methods. But parts of the system need to do that internally.

Just like the passwd command allows normal users to change their own password only, the MethodFinder.findMethod() is supposed to only allow trusted code to call arbitrary methods.

So far, everything is fine.

Class loading exploit

ClassFinder.findClass() is such a trusted method. It loads additional classes with the the privileges of the calling code. Just like passwd lets you change your own password.

But in case of an error, it tried to recover by loading the class with full permission. If we think about passwd, a similar bug would cause passwd to change the password of root instead of the current user.

The operating systems will allow passwd to change the root password, just like Class.forName() allows ClassFinder.findClass() to load classes in the wrong security context.

Method invocation exploit

This one is a bit more complex. The trusted method is Method.invoke() here.

But there was a second trusted method involved called MethodFinder.findMethod(). To stay with the operating system analogy, think of it as a shell script that is run with root privileges via sudo.

This method/program does not validate it's parameters, it just passed them on to passwd. Now passwd is invoked in a trusted context. Therefore it will happily change the password of anyone.

8
  • I still don't understand. I'm not asking for a description of the exploit. I'm asking about the special APIs that use weaker checks, and why they were written to use weaker checks. ClassFinder.findClass() is not listed in the Secure Coding Guidelines as a special API that uses weaker checks. In contrast, Class.forName() is one of the special APIs that uses weaker checks. So, I don't care why ClassFinder.findClass() is buggy in how it invokes Class.forName(); that's not what I'm asking. Rather, I'm asking why Class.forName() uses just the weaker checks.
    – D.W.
    Sep 1, 2012 at 18:11
  • To put it another way: you didn't answer the question I was curious about (you answered a different question). I'm not asking why trusted code has bugs; of course, code always has bugs, and trusted code that invokes special APIs is not immune to that. Rather, I'm asking why the special APIs invoked by the trusted code are specified and implemented to only do the weaker permission checks. If those special APIs had used standard permission checks, it would render an entire class of bugs in trusted code irrelevant, and stopped the exploit. So why don't they use the standard permission checks?
    – D.W.
    Sep 1, 2012 at 18:26
  • @D.W. As I said, those special APIs use weaker checks because they are required internally by trusted code, that is invoked by a less privileged context. It is exactly the same reason why the Linux kernel allows the passwd command (marked as trusted by "setuid root") to modify /etc/shadow although it is called by an unprivileged user. In the Java environment, all classes belonging to the JRE are trusted, this includes ClassFinder and MethodFinder. So it is their responsibility, to pay special care on invocation of methods, which do not use the full calling stack for permission checks. Sep 1, 2012 at 21:49
  • @D.W. I tried to make my posting clearer. Sep 1, 2012 at 22:01
  • 1
    @D.W. Class.forName() use weaker checks because it is required internally by trusted code, that in turn is invoked by a less privileged context. Sep 1, 2012 at 23:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .