As a fundamental of security engineering/incident management, if your system is compromised, You must rebuild your system. Because now attacker has enough details about your system. Even if you change encryption key, algorithm, password he still be able to attack again. Because you don't know how he compromise password.
If you change password or key, still he will be able to access it because i assume that you have no idea about attack. What you know is result.
Identify vulnerability and try to resolve that particular issue.
Therefore rebuilding your system is required with additional security controls. Don't try to achieve security by obscurity.