I’ve designed a custom network protocol using high-level LibSodium primitives for authentication and encryption. The purpose of the protocol is for a game, and the security is to ensure only paid players can connect and play. I’m offloading initial authentication to a webserver over HTTPS, so my protocol is merely transferring the client from the HTTPS connection with the webserver to a connection on a backend dedicated game server. It is important to ensure only authenticated clients are in the game servers to save money on server runtime costs. No sensitive personal data is ever touched or seen by the client or dedicated servers — the only risk is server runtime.

I’m now at the point of looking for a way to verify the security of the design. I’ve done my best to research many kinds of attacks, such as DoS, MitM, replay, reflection, and ip spoofing attacks.

What kind of steps would you recommend, beyond self-study and self-verification? The protocol is fairly simple, but perhaps a little too long to post here directly.

Here were some pragmatic ideas I had:

  • Visit local university to find security professors who are interested in answering questions.
  • I work at Microsoft - I could potentially look to make friends with security engineers here.
  • Third party security audit - I have no idea if these are respectable or even affordable.
  • Simply release the game in the wild - I have low-risk in the event of security compromises, and can potentially patch vulnerabilities whenever they are detected from actual attackers.

How would you go about verifying the design of a custom protocol with similar use-cases?

  • 1
    Visiting professors, making friends, and third party security audit are all the same thing. They differ only by the details in how to engage or pay. We cannot advise on the cheapest way to do something. – schroeder Apr 3 '19 at 9:03
  • You’re right! Though I think listing ideas for strategies to engage is something readers would find valuable (like myself)! Not for the purpose of finding the cheapest solution, but simply to get the ball rolling for newcomers to the topic. – user203621 Apr 3 '19 at 9:34

As with everything, you engage the control that is proportionate to the risk. We can't really advise on the appropriate control without understanding more about the risks.

These are a few processes that are commonly used:

  • 3rd party review (be it professors, friends, or paid reviewers)
  • threat modelling/vulnerability modelling (standard frameworks exist for this)
  • researching similar protocols to see what threats/vulnerabilities they identified and how they countered them in their design
  • agile, reactionary design ("just putting it in the hands of users")

The last point is a valid option, but you have to employ it in such a way that respects the risks/costs involved. Will you be reactive enough to mitigate problems? In the worst case scenario, what are the potential impacts? Will you be able to identify problems or will vulnerabilities be exploited "under the radar"? You also have to be aware of any ancillary impacts that you might not be accounting for, like the impacts to user's private data, etc.

And all that said, the same considerations apply to all the other points. If you miss something in a review, modelling, or research, what will the impacts be? If you are comfortable with those impacts, then use those controls. If you are not comfortable with those impacts, then you need to use the control that provides better assurance that your risks are known and properly treated.

  • What kind of threat modeling frameworks can readers search for? An example or two would be helpful, to get pointed in a good search-engine direction :) – user203621 Apr 3 '19 at 9:37
  • Microsoft has a couple of well-known threat modelling processes. – schroeder Apr 3 '19 at 9:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.