Typically, you have two vectors here.
- Loss of the Private Key
- Unauthorized use of the Private Key
Generally, some hardware backing is recommended to address #1, such as an HSM, SmartCard, or other physical crypto device.
These devices also typically have some from of access control, ranging from IP ACLS with passwords, all the way down to a pin with small retry counts.
As a general rule, you need to be restrictive on what can access the PK for signing, for example, if jenkins builds your software, then signs it, only allow jenkins processes to access the PK, additionally all signing could be performed on a seperate jenkins slave, which is as isolated as possible from everything else.
However, as we all well know, damage control is almost as important as the attempts made to prevent damage. One should ensure he/she has a way to revoke the signing key should it become compromised, and plans to replace and transition cleanly.