When you call
VerifyData(), you must specify which hash function to use (because signatures operate on hashed messages). Moreover, with RSA specifically as described by PKCS#1, the hash value is encapsulated inside a structure which contains the object identifier (OID) of the hash function. The invoked method must thus map the provided hash function specification (which can be a string) to the corresponding OID, and, in the Active Directory, this must go through the AD server (the AD server is a kind of gatekeeper for such mappings).
When the calling code runs as a local account, the AD server may refuse to respond, hence the delay.
This is a misfeature of the .NET implementation: for hash functions specifically, the code should know of the OID of the hash functions it implements (there are not so many) and should not need to talk to the AD server for that (these OID are "well-known" and standardized, they cannot change at the whim of an AD administrator).