Embed Private Key Inside Agent [closed]

Question

We are owing a domain with wildcard certificate (*.mycompany.com). Now we have an application running as an agent on the customer machine. The browser is communicating with the agent by sending websocket requests. Now, we are moving to WebSocket Secure (WSS). Since browser communicates with the agent by sending wss requests to the agent, so we can call the agent a web server. In order to make sure the wss request is not tempered, the agent need to validate the certificate. For validation agent need our wildcard certificate's private key which we cannot share. Now, we are purchasing a specific sub domain certificate from a CA say newdomain.mycompany.com and will embed the private key newdomain.company.com inside the agent. Now I am thinking some malicious user might misuse our new subdomain (newdomain.mycompany.com ). What is best way to mitigate this issue.

Note: I am developer not security person.

Answer 1

Security-wise, assume that any cryptographic keys you put in your client binary can and will get reverse-engineered and end up on pastebin.com.

The first question is what authentication goal you are trying to accomplish; ie is each instance of your agent tied to a single user account on your service, or are you just trying to ensure that only your agents can connect to your service (ie some kind of DRM)?

If your goal is either to tie an agent to a user account, or just to protect traffic between the browser and the agent, then I would design a "registration protocol" that registers a unique public key with the server for each copy of the agent. Such a registration protocol might look like this:

• During install of the agent, have the agent reach out to an enrollment endpoint on the server (possibly having the user logs in to their account (username / password / MFA) if that makes sense)
• The agent creates a new keypair and certificate signing request (CSR) locally
• The agent sends the CSR up to the service and the service signs and returns a client certificate.

This would require you to run your own private CA and configure your server to accept client certs from that CA. A bit tricky but not as bad as you might think. If and when this gets reverse-engineered then the attacker has only compromised their own copy of the agent and all other instances of your agent remain secure. Note that this model will strongly protect against man-in-the-middle traffic interception attacks and against reverse-engineering, but will not prevent someone from writing their own agent code and registering it.

If your goal is DRM (ie ensuring that the client is an authentic copy of your app), then something like what you've described is kinda your only option: embed a private key in the agent that's common to all agents. Be aware that this is a difficult problem to solve; if and when this gets reverse-engineered, attackers will have fully compromised your DRM key. So you will need to obfuscate the key to make it difficult to reverse-engineer. This is a cat&mouse game with obfuscation techniques needing to constantly evolve to stay ahead of reverse-engineering techniques, so you'll likely need to purchase a commercial obfuscation tool and have a plan to update your obfuscation algorithm and rotate your client key periodically (both for new and existing copies of the agent) to force attackers to start over with the reverse-engineering effort.