Whenever I read about a cybersecurity competition, I always wonder, since almost everyone involved is a security professional, can such exercises ever be realistic? One of the weakest links in any information system is the user; how can a competition simulate the effects of social engineering? Do they find a bunch of random people to act as network users? Do they add in confederates who intentionally click on everything?
Can you give an example of a "cybersecurity" competition? Are you referring to Capture The Flag competitions?– Lucas KauffmanJun 28, 2013 at 5:53
Yeah, they don't really handle SE in that regard, they'll handle it as answered by Terry below. They don't have users in the environment that will click on stuff. I have seen some CTFs (especially oCTF at Defcon) that will award points by doing things like sneaking into the organizer area and grabbing a flag there, or spying on the other teams. There are no SET-like challenges, since it mostly requires an uneducated user.– g3kJun 28, 2013 at 15:15
I am not certain you understand the common types of security competitions. Most are specifically for security professionals, and have nothing to do with what a non-security-aware user would do.
Some examples: working out how to exploit rare or hard-to-find vulnerabilities, linking different vulnerabilities together, writing zero-days, identifying systems from limited information etc.
Simulating the effects of users randomly clicking on things is mostly irrelevant to those competitions, and is not what I would consider social engineering anyway.
That said, some competitions allow social engineering as a technique, giving you points for info/trophies gathered by non-technical means (ie lying, cheating, physically gaining access etc)
The point of those competitions is not perform an attack like if it was real where you can trick the user to give away data, rather it is a competition that shows who is the fastest team finding specific data (flag) using technical means.
As an example, look at it as if it was an online wargame (hackthissite, hackthis...) but played as a team.
One particular CTF I participated in, Air Raid organized by ThinkSECURE, awarded "social engineering" in two ways.
The first way is giving out points for obtaining the participant tags of other teams. Every team member is given a tag at the start of the competition. The tag is needed when claiming points from the game masters for completing certain task. Obtaining the tags of other teams using any means (besides physical of course), will earn your team a decent amount of points.
The second way is giving out points for catching other teams breaking the rule of the game. This requires some rather strong evidence but is worth a good amount of points if you successfully report another team.