The important part is the private key to which the certificate is linked. Unfortunately, some widely deployed documentations get it wrong and call "certificate" what really is the union of the certificate itself (which contains the public key and is entirely public) and the corresponding private key (which had better not be public, of course).
In order to keep the private key private, it is best not to copy it in many places. When a private key "travels" between computers, it is normally as a PKCS#12 archive (previously known as "PFX"), which contains the certificate and the private key, and is password-protected. The entropy of this password is crucial for the security of this transit, but, of course, this protects only the key as it is transferred. You are right: the more a private key is copied, the more vulnerable to theft it becomes.
The normal way of doing this is the following:
Each build machine has its own development certificate. Development certificates are certificates which you generate yourself, with a custom CA (which can be some crude usage of the OpenSSL command-line tool). Test machines, on which development is performed, are configured to accept this custom CA as trusted (or maybe signature checks are deactivated altogether).
For production releases, the binaries which are to be signed are transported to a specific, protected machine which contains the real signature key (the one corresponding to the certificate that you bought from the "trusted provider"). Releases don't occur often, so this manual procedure should be acceptable.
Having a free-for-all signature system, accessible and routinely accessed by developers, looks somewhat dangerous to me. A signature with a key which is trusted by third parties is a responsibility: it makes you liable. You should strive to sign things with that key at rarely as possible, i.e. only for releases.