There is a unique association between public and private key. That is, if the sender uses a certain private key to sign a message and you verify the signature using the corresponding public, then the signature verification will succeed only if the message has not been altered.¹
The verification procedure and nature of the association between public and private varies with the cryptosystem you are considering (RSA, DSA, etc.), but the statement above holds true for any asymmetric scheme.
What really matters is that the sender is the only one that can produce a valid signature because he/she is the only one who knows the private, but anyone knows the public, so anyone can verify the signature.
Upon signing, GPG adds a token to the text message which can be used to verify that the message has not been altered in transit: that's the signature. You don't need GPG to read the message because the text itself is not encrypted, there is only an extra token, which could be either a radix64-encoded blob at the end of the message or a text attachment with a similar structure.
GPG does not directly sign the message², it signs a cryptographic hash (SHA-1 or SHA-2 usually) of it. What happens upon verification is that the signature is verified using the public key of the sender to make sure the received hash was actually originated by the sender. If the hash calculated by the sender is considered authentic, it is compared with the hash calculated by the recipient. If both phases succeed, then the message is correctly signed.
[¹] actually, there is a theoretical possibility to break this rule by generating a so-called hash collision. That is to say, a different message with the same hash (see the last paragraph). Collisions have been generated for MD5, while finding one for SHA-3, SHA-2 and SHA-1 is considered unfeasible, although the security margin over the last one is becoming so thin it's advisable to stop using it now.
[²] for some cryptosystems, like DSA, signing a hash is the only possibility.