Hypothetically, should I be concerned if someone has my public key and knows my passphrase? Will they be able to decrypt messages sent to me?
Should I be concerned if someone has my public key and knows my passphrase?
The passphrase is used to encrypt the private key. An attacker cannot do anything with the passphrase and the public key; but make sure he cannot get hold of the encrypted private key. The private key's security is somewhat like two factor authentication: something you own (the private key file) and something you know (the passphrase).
Changing the passphrase might help preventing the attacker gets hold of a copy of your private key encrypted with the already known passphrase, also consider backups. If you're in doubt whether the attacker also got hold of the encrypted private key (which he's now able to decrypt), consider revoking your key.
Will they be able to decrypt messages sent to me?
Absolutely not (without a private key as described above). All decryption operations are performed using the private key.