REJECT is nicer to other people: if the connection is the result of an honest mistake (a configuration error) then whoever tried to initiate a connection receives the standards-approved response that nobody is listening on that port. With DROP, nothing is returned, so whoever is trying to connect will wait until it gets bored.
While REJECT seems better for troubleshooting, a case can be made about how DROP is actually better for tracking bugs. Indeed, imagine a client system that can connect to several servers, until one is found that responds successfully (in a way similar to DNS servers). If the first server in the list is misconfigured, but a REJECT is used, then human users will not notice anything: every attempt will try that server and be promptly responded to with an error packet, and the client system immediately connects to the second server. Such a situation can be in force for a long time, since the human user sees nothing wrong. On the other hand, with a DROP policy, the misconfiguration shows up as a timeout delay for every request, which will induce the human user to investigate.
Indeed, in my daily job, where I must regularly investigate such issues(*), the delay coming from a bad DNS configuration somewhere is often an invaluable tool to pinpoint the problem.
For a personal PC, an additional technical point is asymmetry. Namely a home PC with some Internet access will usually get a lot more download bandwidth than upload (that the 'A' in ADSL). When someone sends to your PC a connection attempt (a TCP SYN), this uses a tiny fraction of your download bandwidth. However, if your PC responds with a rejection (a TCP RST), then this response will use a bit of your upload bandwidth. This opens the possibility of an easy denial-of-service: if you have 10 Mbs download and 1 Mbs upload bandwidth, and you use the "nice" REJECT, then the attacker only needs to send you for about 1 Mbs SYN packets to saturate your network, whether he would need to send ten times as many if you DROP.
In that sense, DROP is probably better than REJECT on a general basis, at least for a home PC. Or, maybe even better, a rate-limiting policy which behaves like REJECT up to some threshold and DROP afterwards (e.g. no more than 10 RST packets sent back per second); I don't know if iptables can be easily configured to do that.
(*) Nominally I am supposed to help design some hairy PKI-related systems, but in practice I spend a lot of time doing this.