The risks: Someone lost the key. If there are more than one person is holding a key, usually the rule 80/20 applies: 80% of the time the door is unlocked by 20% of the people. One can lose the key and not notice for long time. He can even forget that he holds a key to that particular door.

Someone obtained the key legitimatelly, didn't get accounted for and didn't returned the key when he had to.

etc, etc, ... actually, there is a lot of ways to mismanage a physical key.

So, depending on the stakes, changing the door lock may be a good security practice.

The risks: Someone lost the key. If there are more than one person holding some key, usually the rule 80/20 applies: 80% of the time the door is unlocked by 20% of the people. One can lose the key and not notice for long time. He can even forget that he holds a key to that particular door.

Someone obtained the key legitimatelly, didn't get accounted for and didn't returned the key when he had to.

etc, etc, ... actually, there is a lot of ways to mismanage a physical key.

So, depending on the stakes, changing the door lock may be a good security practice.