A potential solution could be to rely on intermediary servers and don't allow P2P communication for untrusted clients.
Let's say it's a chat program (like Skype), the client at the same time becomes an intermediary server on the network and is able to relay data between other clients.
For someone to contact you, he first must look up the IP of the intermediary node(s) you are connected to (through a DHT or something similar) and then send everything they want to say to you to that node, which will relay it to you. No possibility of DoS here as bringing down the node won't do any good and your client will immediately reconnect to a different one.
Once that data reaches you (through the intermediary), let's say it was a contact/chat request and you accepted it. At this point, you trust that person to not do anything malicious and give them your IP (through the intermediary as always), and then he connects directly to you. Also, if it was a contact request, your client also saves their public key which means the next time the person goes online this entire process will be done automatically and your client will give them your IP right away (through an intermediary as always).
This can eventually be thwarted by flooding the network with malicious clients (which will act as intermediary nodes but won't respect the confidentiality of other node's IPs) but once the software gets popular and there are enough legitimate nodes it'll be very hard to do that (think about a sybil attack on Tor).