I have an app installed on customers' machines, a server, and a regular website between the two. In order not to have the customers configure anything (user credentials, proxy settings...), the app communicates with the server through the browser.
To prevent any random website to communicate with the app, we have decided to implement the following automatic challenge mechanism, using asymmetric public/private encryption.
For simplification purposes, I will not talk about the browser, but keep in mind that it's the browser that passes messages between the app and the server; they both don't know directly about each other.
I have generated a public/private key pair for both the app and the server. Each one knows about the other's public key and its own private key. The app private key will be deployed on the customer's computers, so it's not "really" secret (it's a business-line application, but it's always possible that the key ends up "in the wild").
- The app generates a random token (a guid) and sends it to the server.
- The server encrypts the token using the app's public key, signs the (original) token using the server's private key, and then sends both to the app.
- The app checks the signature against the token (using the server public key), and checks that the decrypted data matches (using its own private key). If they both match, the server is authorized to do futher operations.
Is it secure enough to make sure that the server is authorized to make requests? If not, how can requests be spoofed/forged, and how can I further secure it?
I have since realized that this scheme does not implement mutual authentication. Since the server is not called directly from the app but only through the browser, and the browser is authenticated on the server (using WCF), is it necessary?