I would like to give a daemon-style process (i.e. no user interaction) access to a shared secret key so that it can access a shared, encrypted data file. User applications accessing the same encrypted data store the shared secret key in the user's OS keychain (e.g. OS X Keychain or Gnome Keyring, etc.). The OS keychain, in turn protects the secret key with the user's login (or other user-specific) password.
Encryption here is used to protect data as it transits public networks between server and client. The data in question is raw data from scientific experiments. Mostly non-valuable (i.e. unlikely to draw an attacker to speculatively break the encryption and browse the data), but some of it is potentially very valuable and the users wish to keep its details private until they choose to share it publicly (i.e. protect it from casual observers on the network). The value of the data is hard to determine without scientific analysis, so watching users' activity (including queries) as they transit the wire unencrypted would provide some clues to observers about a datum's value.
The daemon-process is a query server for the system. It reads from the shared data, processes a query, and returns identifiers for a result set to the client who then pulls the identified data from the shared store.
We allow synchronization of the encrypted data to users' local workstations/laptops, so encryption of the data on disk is at most a nuisance to a determined attacker. We also allow users to run the query server locally (still started by init.d/launchd), so I want to do whatever I can to protect the secret key stored with the daemon. The major risk is that a determined malicious user is able to discover the secret key from the daemon on their system. Exposure of that key would require us to change the key, update all users' and then re-encrypt the database.
What is the best way to store the shared secret key for the daemon process? I can put the key on disk and encrypt the file on disk, but the daemon process must then have the key to decrypt that file. This seems equivalent to just having the shared secret key for the original encrypted data. My naive approach would be to store the key in the application's binary, but I can't imagine that's the best option. Any advice?