If I understand you correctly, "no one else" means in your case "not my team" (so persons with access to source code, servers etc.).

So I see only one solution:

1. Extract the signing functionality (create a signing service/server).
2. Run this in an dedicated environment/server/vm (store the key here).
3. Restrict access to this as required.

If I understand you correctly, "no one else" means in your case "not my team" (so persons with access to source code, servers etc.).

So I see only two solutions:

Option A:

1. Extract the signing functionality (create a signing service/server).
2. Run this in an dedicated environment/server/vm (store the key here).
3. Restrict access to this as required.

Option B:

Restrict access to your current server/service.