The problem: A single broadcaster needs to send a series of messages through a peer-to-peer mesh network such that only specific other pre-approved nodes can read them. The set of authorized receivers may be very large (so re-encrypting each message once per recipient is not viable), and may change over time, and there is no guarantee that messages will be received in order.
I have come up with what I think should be a viable protocol, but if possible I would like to Not Roll My Own when it comes to the protocol level as well as the cryptographic primitive level, so is there an existing trusted protocol that meets these requirements?
If not, are there obvious problems with the following broad overview?
Both senders and receivers are identified by persistent public keys, and a multicast group is defined by a shared symmetric secret master key.
To multicast, the sender
- uses the shared master key to generate an encryption key and an HMAC key;
- creates a message by encrypting the payload and appending a public sequence number and a copy of his public key as an identifier;
- generates an HMAC with the generated shared key and a signature with his persistent private key;
- hashes the master symmetric key to get a new master key for the next message.
Signed-and-MACed messages are percolated through the network with the majority of the nodes simply passing them along; authorized receivers, however, upon seeing a message tagged with the public key of a multicaster they are listening for, at which point they can re-hash their copy of the shared symmetric key as many times as needed to get the master key state corresponding to the sequence number, thus allowing them to verify and decrypt the message.
Hash-ratcheting the master key is an idea taken from the SCIMP protocol to ensure forward secrecy.
To add a member to the receivers group, a single two-way secure exchange is performed between the sender and the new receiver to transmit the current state of the master symmetric key.
To remove a member, the group must be rebuilt--I don't see a way to avoid simply performing a series of one-to-one exchanges with each remaining member of the group to distribute a new master key.
Similarly, if any of the receivers are compromised, exposing the master key, the group just has to be rebuilt. I don't see any way to produce automatic healing for future secrecy like, e.g., OTR does without doing individual exchanges for every message, which is precisely what I'm trying to avoid.