Well, besides this being a tremendous privacy concern you must worry about the effects it will have on your security. It could be something as insignificant as an attacker calling your phone attempting to social engineer you (with these ID's, it's possible that an attacker could impersonate an authority you trust because 'how else could he know these things?') to them being able to guess your security question(s), I believe 'mothers maiden name' is one of the most commonly chosen security questions.
I suppose if you really pissed someone off (who knew what they were doing), them simply having your address will give you a bad day (swatting anyone?).