I don't know enough about C#'s serialization to know whether it is a security risk, but I can tell you this:
In Java, it is not safe to unserialize untrusted data. There are a number of subtle security pitfalls that can really screw you over. If you are guru-level, you can probably avoid the pitfalls, but an average developer might be in danger.
Here are some examples of subtle points:
If you want to deserialize untrusted data, you have to write special deserialization code to defend against, e.g., a malicious byte sequence that defeats your code's security invariant.
If you do any security checks in your constructor or factory methods, you have to duplicate them in special deserialization methods. If you call the SecurityManager in the constructor or factory methods, you have to replicate those calls in a special deserialization method.
By the way, this example illustrates subtle and dangerous interactions between deserialization and the Java permission model (based upon stack inspection).
There are some crazy hairy security pitfalls associated with deserialization that (1) allow an attacker to mutate fields that should not have been mutable, or (2) allow an attacker to exploit concurrency/reentrancy bugs to break security-critical object invariants or learn secret values you shouldn't be able to learn, or (3) observe incompletely initialized objects (thereby allowing to mutate objects that were intended to be immutable).
Here are some more insane risks with Java deserialization.