First thing to do is define what you mean by "secure and authenticate". You need to put into words who the attackers are, what they can do, and what they should be prevented from doing.

On **example** of such an attack model is the following:

 - Attackers are outsiders. They eavesdrop on the common channel (the radio waves). They want to obtain the confidential data that the group members exchange with each other. The group members trust each other.

If that is the attack model, then it suffices to come up with a shared secret _K_, that all group members but no attacker knows, and then encrypt packets with some symmetric encryption algorithm (say, AES in a suitable mode) using _K_ as key. Initial distribution of _K_ can be done by having each group member own a RSA key pair; one group member generates _K_ randomly, encrypts it with the public key of each group member, and broadcasts that.

Now let's extend that attack model into **another one**:

 - Attackers are outsiders. They eavesdrop on the common channel, but will also attempt to send fake messages, sometimes replaying previous messages, with or without alterations. Such fake/replayed message instances shall be reliably detected and dropped by group members.

Now you also need a [MAC][1] for each packet (so that the attackers can only replay previous packets, not create new ones or alter existing packets), _and_ some sort of connection context so that [replay attacks][2] are detected. This can be done by embedding a sort of time stamp or counter in each packet: each participant remembers the counter values for the last packets received, rejecting packets which have already been seen or which have a "too old" counter value. Such mechanisms are often called _window-based_ and there is some balance to strike between replay attack prevention, memory requirements (to remember past packets), and tolerance to packet loss due to the unreliable nature of the transport medium. Peruse [DTLS][3] as an illustration of how the SSL/TLS protocol can be adapted to packet-based unreliable mediums.

If the attacker is allowed to try to send fake messages, then he might also be able to advertise a fake RSA public key. Distribution of _public keys_ is a matter of [Public Key Infrastructure][4], with _certificates_ being the usual solution.

Maybe we **want more** ? E.g., what about the following:

 - Group members are attackers too. Each group member will try to eavesdrop on and/or interfere with one-to-one communications between other group members.

In that case, you will need pairwise key exchanges. For any pair of group members _A_ and _B_, there must be a shared secret _K<sub>A,B</sub>_ that other group members and outsiders do not know. This can still be leveraged from pre-distributed RSA Key pairs: when _A_ wants to talk to _B_ and they don't already have a shared secret (or they forgot a previously shared secret, e.g. _B_ rebooted his computer and that shared secret was in RAM only), then _A_ generates a new random key _K<sub>A,B</sub>_ and encrypts it with _B_'s public key.

There again, read up the complete [TLS][5] and [DTLS][6] standards: they are quite readable, and that will show you what you have to think about.


  [1]: https://en.wikipedia.org/wiki/Message_authentication_code
  [2]: http://en.wikipedia.org/wiki/Replay_attack
  [3]: https://www.rfc-editor.org/rfc/rfc6347
  [4]: http://en.wikipedia.org/wiki/Public_key_infrastructure
  [5]: https://www.rfc-editor.org/rfc/rfc5246
  [6]: https://www.rfc-editor.org/rfc/rfc6347