Changing your e-mail and phone number is silly. Your phone number, unless unlisted, is a matter of public record and easily discoverable. Even if unlisted, it is still a publicly shared identifier that can be discovered with some investigating. Your e-mail address is also a public identifier and can be discovered with some effort.
Having identifiers in the hands of an attacker is not an immediate threat as long as they don't have access to those identifiers. An attacker knowing your phone number doesn't allow them to use your phone. Similarly, if an attacker has your e-mail address, as long as your password to your e-mail address is secure (change it to be sure) then they can't make use of your e-mail address either.
Both of these can be spoofed, through either caller id spoofing or e-mail sender spoofing, but such spoofing is easily detectible to anyone who knows what to look for (and any high threat target should be aware of these things and check them, or simply have procedures built in to verification to mitigate this.)
When you are comprised, the important part is to remove access from the attacker. Credit card and bank account numbers change because they are used without any authentication, so they act as both identifier and authentication, thus they have to change. Anything that requires a separate authentication (even if simply the physical access) doesn't have to change as long as the authentication can be assured to not be compromised.