There are only 100000 potential values if you have a 5 digit PIN. That means that a single PIN can be brute forced in a relatively small length of time, unless there is some form of secondary restriction against a person standing next to the door and trying each number. This gets worse if there are multiple people with distinct codes for access to the system - assuming each person selects a random code, an attacker entering a single code has a n/100000 chance to find a working code, where n is the number of people with access.

It would probably be reasonable to expect a dedicated attacker to manage 1000 tries per hour - it doesn't take very long to type a 5 digit number. That would give an upper bound of 100 hours, with a single code. That's 3 (pretty boring) weekends with some breaks for food, which, depending on what is in the building, may well be worth it. The system as described _can't_ implement account lockout - the only way to determine which user is trying to access is through the code.

So, how to solve this? 

 1. Use the PIN as a secondary factor - have an access card, and a PIN, for example. It is then possible to have account lockout after some failed PIN entries. 
 2. Enforce limits on the PIN in other ways - have someone guarding the door who prevents too many tries, have a CCTV camera pointing at the door which is monitored for unusual activity (someone standing there trying each possible code)
 3. Have a much longer PIN, making it impractical to keep trying codes

You would also probably want to ensure that the codes being entered cannot be monitored in other ways - for example, by pointing a video camera at the key pad.