I'm designing the update procedure for IP Camera's. I'm wondering if my current design is secure enough.
I have a binary file that I want to get to a device. I make a hash of this binary file and encrypt the hash with my private key (a signature). I put this binary file and the encrypted hash in a zip file. I create a MD5 hash of the zip file. I put the public key, MD5 hash and zip file publicly available.
I have multiple devices, some of them under hostile client control, some of them under friendly client control.
The friendly clients have a download algorithm like this:
On receiving a command, containing URL's, the provided URL's are used to download the public key, the MD5 hash and the zip file. The downloaded zip file is checked to see if it matches the MD5 hash. It is then extracted. Then the encrypted key is decrypted with the provided public key. A hash is made of the binary file and compared with the decrypted key.
If both the MD5 check and the hash check of the binary file succeed, the binary is executed with root access.
The hostile clients have access to the whole device, so any stored secrets on the device are known to them. They don't have access to the system I store the private key on, though.
From what I can see, the command and timing thereof is potentially hostile, the URL's are potentially hostile, so the zip is potentially hostile, as is the binary and hash and MD5. The MD5 hash is easily bypassed, but it only serves as file integrity check. The encrypted hash of the binary, however, is my main point of defense.
When the zip and the binary and the encrypted hash and the public key are all under control of the attacker, is my system still secure (that is, can an attacker make a friendly client execute a binary that didn't originate from me)? Is breaking the security easier since the "ciphertext" and "plaintext" and the public key are provided? If it is easier, is it easier by any relevant magnitude?