As in always in security, what threat are you attempting to protect against?
It seems from the question you are worried about availability. Typically a hash table will have limiting performance of O(1) for simple operations, but degrade to worst case of O(n). (See Secure Coding Guidelines for Java SE .) Say, will a web server use resources disproportionate to the size of a maliciously-crafted worst-case request. The linked article in the questio is about Java Serialisation, which is a hole different thing (and really doesn't protect availability).
It's all very much up to the implementation.
Going back a decade or so, the Sun implementation of
HashMap did generally just use a modulus of whatever came out of
Object.hashCode. See for instance, the definition of
String it is trivial to generate different text with the same hash. Give an old
HashMap a bunch of keys with the same hash, they'll only use one bucket, and the performance will be terrible.
Later the Sun implementation mixed the hash value around before taking the modulus. However, if the hash was the same to start with, it'll still be the same.
TreeMap solves the issues, but the benign case performance isn't the best, though has improved.
More recently, to placate those who don't know how hash tables work, OpenJDK used variants of MurmurHash, with a per-instance random seed, for
String when there are many collisions. This replaces
String.hashCode - simply adding the same number to a fixed hash wont alter collisions. Although technically "non-cryptographic", it is supposedly difficult to generate collisions without knowing the secret seed. There are always side-channels.
Now replacing MurmurHash, is the obvious algorithm of using a tree in place of a linear list for buckets when there have been many collisions. As
HashMap was never designed to this, it's an outrageous hack, but a hack where the library comes up smelling of roses even when abused (mostly). The alternate algorithm only comes into use when there are many collisions (as with Murmur) and only for instances where every key appears to be implementing