Language safety is not clearly defined while there're warnings about for instance Java. So how can you say that language is not safe while language safety is not clearly defined? If Java is unsafe now, is Java version 7 "safer" than Java version 2?
A language can be said to be "safe" if it is possible to implement its interpreter/compiler/runtime environment efficiently, in full respect of the language specification, and still forcibly preventing "bad things" to happen. Then it depends on what "bad things" you are talking about.
Java is "safe" if you want to protect the machine from the code written by the developer. Array bounds are checked, types are unescapable, and memory deallocation is handled by a garbage collector, which means that it is not theoretically feasible to make a Java applet, however poorly written, execute attacker-chosen native code. In the same conditions, C is "unsafe" because it cannot be fully implemented with checks which prevent arbitrary code execution.
Of course, in practice, things change a bit: C code can be made "safer" with various techniques (like DEP and ASLR) and the OS can "contain" native code under some specific access rights (a "normal user" process is not
You cannot have a one-size-fits-all definition of language safety, unless you are content with a subjective slogan (COBOL sucks ! F# rules ! Prolog forever !). It all depends on the context.
There may be some confusion here. A programming language does not inherently include vulnerabilities. It is the programs written in those languages that do.
In the case of Java, there is a sandbox which has not proven to be entirely effective. This sandbox was intended to allow a program to be written in Java, and then to run in the browser, without having access to the computer it was running on. It is the Java sandbox that has had security problems, not Java itself. I cannot speak to the comparison between Java 7 and Java 2.
Languages are classified as safe and unsafe based upon a few properties. A safe language as you say include at least the following properties:
Java language has all these properties and hence can be easily classified as a "safe" language. The insecurity you mentioned is not because of the language itself but due to insecure sandbox environment and insecure library calls. Take apart any recent Java exploit code and all it does is bypassing security restrictions to call privileged class code. It has nothing to do with the insecurity of the language itself but with the complex and insecure design of the java libraries.
Language safety can be interpreted more broadly than affecting merely security. But as this is a security site, let's just restrictive ourselves to security.
Clearly language selection has a significant on whether you will get certain types of vulnerability. Write a non-trivial program in C and it will be full of memory corruption errors (buffer overflow, use-after-free, etc).
There a couple of useful definitions:
Memory-safe - This means a program within the language cannot access memory outside of allocations. No running beyond a buffer or using a allocation after it has been freed. That's tricky to automatically prove for a language like C. Memory-safety is typically achieved through garbage collection and using the more restrictive type-safety.
Type-safe - Don't confuse this with just not messing up your types in unsafe languages. This is a guarantee which goes beyond memory-safety in that you cannot use a type in a way inappropriate to the actual type of the object. You can't treat a pointer as an integer, for example.
Interestingly, from 1.5 Java is not vanilla type-safe. It is type-safe with respect to raw types, as used by the JVM and Java language 1.4 and earlier. It is unsafe with respect the full language type system including generics. This doesn't seem to matter at all, largely because in order to access an object the bytecode must perform a raw type check.
For non-memory corruption vulnerabilities libraries become more important. For instance, injection is an incredibly popular class of vulnerability, but there's not really any excuse in having any at all.
Mobile code (that is code that is mobile and untrusted) is an altogether different thing. Safety can be achieved via a sandbox (as Google's NaCl), an Object-Capability Model (jargon essentially meaning OOP in a type-safe environment) or some combination of the other two.
As for Java SE 1.7 vs 1.2. There hasn't been much change. 1.2 is very different to 1.1, and 1.0 was a bit different again (although really before I was paying attention to these sorts of things).
Specific issues in the Java language which do cause vulnerabilities: