Creating a digital signature for a message involves running the message through a hash function, creating a digest (a fixed-size representation) for the message. A mathematical operation is done on the digest using a secret value (a component of the private key) and a public value (a component of the public key). The result of this operation is the signature, and it is usually either attached to the message or otherwise delivered alongside it. Anyone can tell, just by having the signature and public key, if the message was signed by someone in possession of the private key. So, how does this work?
To use textbook RSA as an example of asymmetric cryptography, encrypting a message m into ciphertext c is done by calculating c ≡ me (mod N), where e is a public value (usually a Fermat prime for efficiency reasons), and N is the non-secret product of two secret prime numbers. Signing a hash m, on the other hand, involves calculating s ≡ md (mod N), where d is the modular inverse of e, being a secret value derived from the secret prime numbers. This is much closer to decryption than it is to encryption, though calling signing decryption is still not quite right. Note that other asymmetric algorithms may use completely different techniques. RSA is merely a common enough algorithm to use as an example.
The security of signing comes from the fact that d is difficult to obtain without knowing the secret prime numbers. In fact, the only known way to obtain d (or a value equivalent to d) from N is to factor N into its component primes, p and q, and calculate d ≡= e-1 mod (p - 1)(q - 1). Factoring very large integers is believed to be an intractable problem for classical computers. This makes it possible to easily verify a signature, as that involves determining if se ≡ m (mod N). Creating a signature, however, requires knowledge of the private key.