What are the best tools available in order to reverse engineer a communication protocol?
This question appears to be off-topic. The users who voted to close gave this specific reason:
- "Questions seeking product recommendations are off-topic as they become obsolete quickly. Instead, describe your situation and the specific problem you're trying to solve." – Jens Erat, Community, Xander, RoraΖ, Stephane
Assuming it is communicating over a network:
- Wireshark and/or Tcpdump to capture and analyse data.
- Hex Editor to define headers, footers and other data structures within the packet streams. A lot of the traffic will most likely not be human readable characters.
- TCPReplay to replay the traffic, allowing you to modify and re-test.
- WireEdit, allows you to modify PCAP (capture files) and save them.
- Reverse engineering tools for the communicating application such as OllyDBG, Immunity Debugger, GDB.
- A text editor to edit your systems host file. Useful to intercept and reply to communication destined to remote servers. Play back the traffic you want and see how the protocol behaves.