If the password is strong, then there is no worry whatsoever.
POP3 does not open new ways for cracking passwords IF you use it sensibly, i.e. within SSL/TLS. Otherwise, depending on the underlying authentication mechanism, a hash of your password may transit on an unprotected connection (bad !), or maybe even the password itself (worse !).
A feature of POP is that the emails are removed from the gmail servers once you retrieved them, so:
- you can no longer read them through the gmail Web interface;
- you no longer benefit from the disk redundancy and automatic backups of the Google infrastructure;
- you can read your emails locally even when deprived of an Internet connection.
Therefore, this is not only a problem of performance and brute-force attacks.