1

We are owing a domain with wildcard certificate (*.mycompany.com). Now we have an application running as an agent on the customer machine. The browser is communicating with the agent by sending websocket requests. Now, we are moving to WebSocket Secure (WSS). Since browser communicates with the agent by sending wss requests to the agent, so we can call the agent a web server. In order to make sure the wss request is not tempered, the agent need to validate the certificate. For validation agent need our wildcard certificate's private key which we cannot share. Now, we are purchasing a specific sub domain certificate from a CA say newdomain.mycompany.com and will embed the private key newdomain.company.com inside the agent. Now I am thinking some malicious user might misuse our new subdomain (newdomain.mycompany.com ). What is best way to mitigate this issue.

Note: I am developer not security person.

5
  • I'm not sure about your design, i.e. what exactly is your application (only the agent?), where exactly is it running (locally on the customer machine or on some machine inside the customers network) and why does the agent needs to use your domain and cannot use a domain owned by the customer? Mar 13, 2021 at 13:04
  • Agent running locally on customer machine. Actually this is our public application. The agent our linked company which reads smart card using wss and send them to server.
    – user960567
    Mar 13, 2021 at 14:23
  • Unfortunately I cannot figure out the answers to what I've asked from your last comment, i.e. it does not actually clarify your question for me. Mar 13, 2021 at 14:31
  • @SteffenUllrich let me clear a little bit. We are partner with a company that creates an agent application(for example dynamsoft web twain agent app). This agent app run on client machine and communicate with the browsers using web sockets. For making it secure we are wss protocol but then the agent app requires certificate for decrypting the traffic coming from browser.
    – user960567
    Mar 13, 2021 at 18:55
  • It is still not clear why you want a to use your own domain for software installed at the customer site. This would also require that the DNS settings of your domain point into the customers infrastructure, which is a strange and fragile setup, since this infrastructure and the management of their internal IP addresses is outside of your control. Mar 13, 2021 at 19:06

1 Answer 1

1

Security-wise, assume that any cryptographic keys you put in your client binary can and will get reverse-engineered and end up on pastebin.com.

The first question is what authentication goal you are trying to accomplish; ie is each instance of your agent tied to a single user account on your service, or are you just trying to ensure that only your agents can connect to your service (ie some kind of DRM)?

If your goal is either to tie an agent to a user account, or just to protect traffic between the browser and the agent, then I would design a "registration protocol" that registers a unique public key with the server for each copy of the agent. Such a registration protocol might look like this:

  • During install of the agent, have the agent reach out to an enrollment endpoint on the server (possibly having the user logs in to their account (username / password / MFA) if that makes sense)
  • The agent creates a new keypair and certificate signing request (CSR) locally
  • The agent sends the CSR up to the service and the service signs and returns a client certificate.

This would require you to run your own private CA and configure your server to accept client certs from that CA. A bit tricky but not as bad as you might think. If and when this gets reverse-engineered then the attacker has only compromised their own copy of the agent and all other instances of your agent remain secure. Note that this model will strongly protect against man-in-the-middle traffic interception attacks and against reverse-engineering, but will not prevent someone from writing their own agent code and registering it.

If your goal is DRM (ie ensuring that the client is an authentic copy of your app), then something like what you've described is kinda your only option: embed a private key in the agent that's common to all agents. Be aware that this is a difficult problem to solve; if and when this gets reverse-engineered, attackers will have fully compromised your DRM key. So you will need to obfuscate the key to make it difficult to reverse-engineer. This is a cat&mouse game with obfuscation techniques needing to constantly evolve to stay ahead of reverse-engineering techniques, so you'll likely need to purchase a commercial obfuscation tool and have a plan to update your obfuscation algorithm and rotate your client key periodically (both for new and existing copies of the agent) to force attackers to start over with the reverse-engineering effort.

Not the answer you're looking for? Browse other questions tagged or ask your own question.