Company does not want any names on phishing reports

Question

We have been recently contracted to run phishing tests for a company. Let's call it a company but basically they are obligated, by law, to assess the security of their environment with phishing campaigns.

We ran our first campaigns not too long ago and the results were pretty bad. Over 70% of their users trusted the "malicious emails" we sent and did whatever the email asked of them.

After it was over, we of course had a out brief detailing our findings. Long story short, they did not want ~any~ Identifiers (email, username, whatever) of who fell for the phish. They wanted "X out of 300" failed to identify the email. Their reason was they did not want to offend anyone. (I wanted to say your customers feelings will be hurt when your employees fall for a potential attack and leak info) I politely accused them of checking a box and not actually being interested in educating their users. They weren't very happy. I should elaborate by saying they didn't even want 2 reports, one showing the emails and another not showing them. I offered it to them because at least they can see how these individual people react to different campaigns overtime. This would absolutely help if user "Sam" clicks on every single link in a email every time over the course of ten campaigns. Surely you would want to educate Sam differently than other users?

My question is, does this not defeat the purpose of phishing campaigns and improving the security of information in your network? Is this even normal?

Answer 1

This initial campaign established a baseline first. So, yes, it's normal. "How do we as a company stand? To what level do we need to train? Do we have, as a whole, secure users or do we have, as a whole, unsecure users?" This report establishes this and the extent to which management needs to engage in phishing training. Were only 5% of users to fall for the phish attempt, then the training focus would be very different. As it stands, now management knows that they have, essentially, a corporate-wide problem and that a phish campaign basically stands a 70% chance of succeeding.

Now, when the company does future phishing training, they can compare results and determine whether the training was successful. "We initially fell for it 70% of the time. This time, we fell for it 68% of the time. It was therefore, not successful." Or "We initially fell for it 70% of the time and now fell for it 50% of the time. We're doing better, but need further training."

Answer 2

No, because by giving names you are assigning blame, security needs to move away from blaming individuals and instead take it as a whole. It's the same as finding a security vulnerability in a web site: you shouldn't blame the developer but should instead look to improve the entire process.

We run phishing campaigns and do not identify users. What we use it for is to identify weakness on our part and that we need to train our staff better. There is no point focusing this training on just a single person.

After a campaign we email all staff, provide the statistics of failures / success, and then provide tips for spotting phishing and how to treat email in general.

Answer 3

I think the correct angle to look at this, is to ask the following question:

With the amount of people that failed the test, what (security) goals would be accomplished if the company had these names?

I would say: none.

---

What is the security goal of a company-wide phishing test anyway?

Typically in every company that relies on IT and has a certain amount of employees, these employees are subject to information security trainings. These trainings mostly cover basic topics like e-mail communication, desktop security and so on. When running a phishing test, management wants to know:

1. if these trainings were successful (as in: worth their money)
2. if any data or IT system that belongs to the company can be compromised due to a lack of good training

If you as their contractor tell them "70% of your employees failed the test", that answers the two questions above. If the management asks for names in a company with 300+ employees, they do not gain any more relevant information and are not doing their job correctly.

The next step is now, to define a new security goal. It should read something like this:

"In the next X months every employee has to participate in a security training. By $month of $year we want $contractor to conduct another phishing test and the percentage of people that fail this test should be below X%."

Would these trainings be more cost efficient, if only those employees had to participate, that failed the phishing test? Probably.
But: you present them to 30% of the company (the ones that don't have to go) as "too stupid to identify a phishing attempt". What this does to morale outweighs all the cost of just sending all your employees to a training. Also: Another reminder for the 30% about information security doesn't really hurt.
There's another reason why this is a good idea: Typically if you run a phishing test, you don't know, why people did not fall for it. Maybe some of them didn't read the e-mail because they were on vacation, sick or just skipped it, because they have an inbox full of more important mails. Nobody can tell you, if they'll pass the test next time. Employees are always your number one risk factor, train them if you can.

Another point I want to mention that was missed so far in the other answers is, that depending on how you communicate the results: most people will know themselves that they failed that test.
You have to inform your employees in one way or another and I assume this is the way that most companies do it: Send a company-wide e-mail with a screenshot of the phishing mail.

"Dear employees, sorry to tell you, but this was a phishing test. There is no free yacht waiting for you. The numbers of people who didn't pass the test were bad, that's why we'll have some security trainings in the near future. A contractor did this for us and we did not collect any personal data, so we do not know who clicked on a link and who didn't. There will be no repercussions. Phishing mails can have really really bad consequences such as... yadda, yadda, yadda..".

People will check their inbox and if it's not too long ago, remember what they did. This will boost acceptance towards a security training and an adjustment in behavior. Invoking fear and pressuring people does no good.

Answer 4

There seems to be some miscommunication about the purpose of these tests.

Using identifiers means finding responsible people, to educate and/or punish them. It might be an explicit parameter of the test that no people can be identified. In many european countries, the local workers council or union representatives would have to agree to such a test and might put this as a condition.

Using statistics means identifying performance indicators. You can measure these against each other to identify, for example, if you are improving or if some awareness campaign you ran was effective. You don't need identified people for this and it might even blur the results.

Finally, the customer pays the bill. You work for them, so while you should point out any professional concerns or thoughts you have, unless it goes against your personal or professional ethics (e.g. the ISACA ethics standards if you are a member), you deliver what the customer asks for.

Answer 5

To answer your question:

does this not defeat the purpose of phishing campaigns and improving the security of information in your network? Is this even normal?

No. If the customer wants a watered down version of the result of the research, it's their call ultimately. But you have all the rights to offer your best advice and give them the power of choice.

Is it normal to be faced with those types of customer reactions? Yes, it may happen all the time, and that could be the consequence of a lot of things. Customer might have their own insecurities (e.g. what if members of management were caught, how to handle that), or might not want to slight their employees, might not know how to handle their training afterwards once the report is public.

If they're the ones paying for it, as their adviser you have to finally honor their decision.

As an extra side note, about something I noticed:

I politely accused them of checking a box and not actually being interested in educating their users.

There's no way to polite say what you just said. But there are other ways to say it without coming across all wrong. Welcome to the world of politics. I see most of the other answers covered the security aspect, so I wish to cover the other subtle political aspect in my answer.

You visibly have a lot of good will and know-how, and your customer appears stubborn from your standpoint for not going to the full length of that exercise.

I found the best way to communicate something like that is to use slightly different ways of saying it, that will promote your cause without necessarily annoying the customer or making the customer feel like you're not letting them call the shots or you're criticizing him and coming across the wrong way.

Here's a few ways you could have said it that would probably have helped promote your vision. It's all politics and can be studied separately.

• Most politically correct way (maximum message dilution): We would be missing out on a great opportunity if we left out that part of the exercise. Are you sure you want to do that Mr. Customer?
• A bit less politically correct but message less diluted and still not coming across off: If we don't go to the full extent and allowing the training to happen where it is due, we would be wasting a lot of energy. Are you sure you want to do that Mr. Customer?

Notice in both cases, I finished up with Are you sure you want to do that Mr. Customer?. That's called the power of choice, put the choice back in their hands at the end of any attempts to modify their behavior or thinking.

If you don't succeed and they still don't want to, let it be. You didn't have the authority anyways, all you can do is advise them the best you can. But still, in a different scenario, you could have affected the customer's thinking and got it your way. But it's not always the case.

Answer 6

I just finished such a campaign (as a customer) and wanted absolute anonymity of the users.

There were a few reasons, among them the most important ones were

• privacy, a complicated matter in some countries
• the fact that I would be sending a global note about the campaign and the results

Your customer may have had other reasons. You gave them the opportunity to have full results, possibly with some advice. They wanted the anonymous version, their choice.

Note: sorry for the typos (or actually - wrong autocompletions by my phone) which made my initial answer rather strange looking.

Answer 7

I totally grasp your desire to capture that data. So anonymize them. The general idea is you take an MD5 hash of their lowercased email address, and grab as many bits of resolution as you need, and either leave them b7R+ or convert to "AOL passwords" like shave-pen-osram.

Forbidden

   John Smith           FAIL
   Frank Frink          FAIL
   Juliana Crain        PASS

What you might be allowed

   Rufus-Castle-Uniform-Enemy    FAIL
   Zion-Lathe-Shoot-Loyal        FAIL
   Flee-Worldly-Variable-Key     PASS

What is safer still

   Bucket Stop-Bad         4/6 failed (66%)
   Bucket Wax-Scissors     5/6 failed (83%)
   Bucket Memory-Egg       4/5 failed (80%)

There are two ways to go as far as anonymity, imagining n bits can contain the number of employees (e.g. employees=700 n=10).

• You can go a few extra bits, like n+6 bits, in which case the anonymization would be reversible and the employee could be exposed: Rufus-Castle-Uniform-Enemy usually hashes out only to [email protected]... Gotcha! There might be a second email, but the more bits, the less likely.
• You can go a few too few bits, like n-3 bits, in which case, the reverse run will reveal Stop-Bad hashes out to jsmith, emccarthy, jcrian, tkido, ctagawa, and ffrink, making retribution impracticable. This winds up creating a group of "buckets" as it were.
• you can salt the MD5, but that fails if the persecutor
  • learns the salt via a brute-force crypto attack
  • simply commands you to turn it over
  • notes the pattern of activities which has been logged, and deduces the user

Your disagreement with their reasons is a classic workplace.se problem, but they may not be telling you all their reasons*. Regardless you must comply in your report to them.

The methods I've provided here with anonymization allow you to present, in your report, the detail data you want to present, while technically complying with their directive. You can either do it in the n+many form, which allows them to backtrack to individual users if they really want to -- or the bucketed form, which does not.

Bucketing is fairly useless at 70% unless you present "In bucket 127, 4/6 users fell for the phish". Bucketing works best when the hit rate is 1/3 the number of buckets or less, so 2 hits in the same bucket are rare. "In 512 buckets, 90 buckets had hits, most likely that's 90-95 people, which is the number you want.

* as a litigator I can think of a really big one. If it were me, I would delete the personalized data "as a matter of routine". Saving everything forever is all fun and games until the subpoena comes.

Answer 8

I agree that the client’s choice seemingly prevents any opportunity for improvement. However, there is another way to improve.

Let all the employees know “We do not know who fell for it and who didn’t, so we can’t fire or reward anyone. However, we do this test every month, and we will publish the percentages. If the 70% is down to 20% by the end of the year, all employees will get a bonus. The size of the bonus will depend on how much below twenty. To help us reach that goal, there will be a weekly e-mail teaching a technique for identifying phishing. Next year, the bonus will be every quarter, but the goal will also be smaller each quarter.”