topic Re: Phishing testing - Consequences of failure in Governance, Risk, Compliance

Phishing testing - Consequences of failure

JK1 — Mon, 09 Oct 2023 09:37:09 GMT

Dear Community,

We have made use of internal quarterly phishing testing for the past 4 years and have an escalation path for failure that follows:

Fail 1: Informal talk from line manager and/or Security and retake phishing course > Fail 2: Formal talk from line manger and retake course > Fail 3: First written warning > Fail 4: Final warning > Fail 5: Potential for dismissal.

In discussions with HR they wish to only consider the tests from the past year rather than the whole history. This would mean either ramping up the number of tests or changing the escalation process.

I was wondering how others approach internal phishing tests and consequences? Do you consider tests over a year old? If not how many tests do you run and what is the escalation process?

Re: Phishing testing - Consequences of failure

tmekelburg1 — Thu, 20 Aug 2020 13:18:52 GMT

The old carrot vs. the stick question...

We setup initial security awareness training for new hires before we allow full access to network resources. If they don't pass they don't get access, easy enough. They can take it as many times as they would like, it's not a fail once and done scenario. Current staff are enrolled into two KnowBe4 interactive video training that take part in the first half and second half of the year. On top of that we send out phishing test emails two times a week. The first time they click on a link or open an email attachment on a test email it gives them a warning and lets them know what they did wrong. The second time they do it, we have them enrolled into remedial smaller 15 minute courses, typically around four extra courses. We've never had anyone get to this point btw. If it did we would cut off access to our client records platform because they are a risk to the company we can't allow.

Security has been built into our culture and that's why we don't typically have any issues. I even tell staff to email us suspicious emails if they have any questions of legitimacy at all. I know it's not advised but I want staff to feel comfortable communicating with us when they have any questions at all. If it's a legitimate email, I tell them so and thank them for being proactive.

Long story short, we use the data from when they first start and until they leave. If we re-hire them, I just pull them out of the archive and keep building on the data we already have of them. Their supervisors are notified upon first enrollment into the program, training they need to take/finish, and any outstanding training needed. I hope this helps, whatever you decide to do make sure this gets into a written policy and approved. Then just always fall back to policy.

Re: Phishing testing - Consequences of failure

CraginS — Thu, 20 Aug 2020 13:28:18 GMT

@JK1 wrote:
...
I was wondering how others approach internal phishing tests and consequences? Do you consider tests over a year old? If not how many tests do you run and what is the escalation process?

Joseph,

Dr. M. Eric Johnson, Vanderbilt U., along with one of my former colleagues Dr. Deanna Caputo, MITRE, have been researching this very issue for years. (I was an unwitting participant in one of their research tests Deanna conducted internally in our company some years ago.) Much of their work looks at using the failures as points to introduce the training, rather than punishment. I recommend you search out their publications for more detail on what they find works and what doesn't.

You can watch Dr. Johnson's 2011 keynote address to the ISSA International Conference in Baltimore, Human Behavior – The Weakest Link? at

https://www.members.issa.org/page/2011ConferenceRecord?PrivacyNotice

where I first learned of his research (and figured out that Deanna had used me in one of their experiments)

To get you started on their recent work, look at

Spear phishing in a barrel: Insights from a targeted phishing campaign
Article (PDF Available) in Journal of Organizational Computing and Electronic Commerce 29(1):24-39 · January 2019

The actual journal source is here, but tat that site you have to pay $51 USD for the paper. the Researchgate link above has it for free.

Here are a few more potentially informative links:

Point-Of-Failure Phishing Training Does Not Work

Going Spear Phishing: Exploring Embedded Training and Awareness Jan.-Feb. 2014, pp. 28-38, vol. 12

I recommend a web search on ["m. eric johnson" "deanna caputo" phishing] to find more journal articles and interviews with them on the subject.

For those who run into horrid paywalls for individual papers in professional journals, be sure to search researchgate.net and academia.edu, where authors often post their works for free. Also, write the authors directly to request copies. Most are happy to send out PDFs for free, to get the exposure.

Next, a suggestion of my own:

Do not even think about using the punishment route you have outlined until you have in writing a statement from the President, CEO, or Board Chair, approved for broad dissemination in the organization, stating that the policy and consequences applies to all employees, including senior managers, vice presidents, and all direct reports to that top official.

Good luck . You have taken on a tough problem.

Craig

Re: Phishing testing - Consequences of failure

tmekelburg1 — Thu, 20 Aug 2020 13:51:15 GMT

Next, a suggestion of my own:

"Do not even think about using the punishment route you have outlined until you have in writing a statement from the President, CEO, or Board Chair, approved for broad dissemination in the organization, stating that the policy and consequences applies to all employees, including senior managers, vice presidents, and all direct reports to that top official."

He should follow whatever his company's process is when drafting, approving, and disseminating new/updated policy but good point on not doing anything until it's officially in policy and communicated out. Thanks for the resources as well!

Re: Phishing testing - Consequences of failure

dcontesti — Thu, 20 Aug 2020 14:16:38 GMT

So it seems (MHOO) that HR has not bought into Security or the risks. Each organization is different and depending where HR resides you might have different routes to take.

I agree with you that there should be a history kept (and HR records are the best place to keep them). I disagree with the HR department that only keeping the information for a year is wrong. I am a firm believe that these types of infractions should become part of the employees permanent file.

HR is probably of the mindset that the tests are not real so no damage was done.

Do you have a specific awareness program for senior management? If not, this is a good place to start. I would use real world stats on things like phishing, virus, ransomware. Once you have buy in at the most senior levels then your plan will work....otherwise its your dept vs their dept.

my two cents

Re: Phishing testing - Consequences of failure

Steve-Wilme — Thu, 20 Aug 2020 15:48:40 GMT

As CraginS says consequences of failure isn't a good way to look at this. Opportunity for improvement and awareness raising in the event of 'near miss' events should work better as a mindset than seeing this as an employee failure. Even CEOs and senior officers can be duped by fraudsters and believe me that's going to be a very difficult conversation series of conversations with employment lawyers if you employ a different policy for senior staff and the rank and file. You really have to do what works.

Re: Phishing testing - Consequences of failure

CraginS — Thu, 20 Aug 2020 16:42:30 GMT

@Steve-Wilme wrote:
... Even CEOs and senior officers can be duped by fraudsters and believe me that's going to be a very difficult conversation series of conversations with employment lawyers if you employ a different policy for senior staff and the rank and file. ...

There is a reason we have the three-level names of phishing, spear phishing, and whaling. Your publicly known seniors are at the highest risk for whaling attacks, yet many organizations allow a culture at the C-suite of "do as I say, not as I do." Once word gets out that a senior who violated policy got a free pass for an infraction that would have been serious reprisal for a worker bee, your program is shot to hell.

Both to fight the culture of allowances for seniors and to hit home on the dangers of whaling, Diana @dcontesti is right, it is essential that you have a custom awareness program for senior staff, and make sure NO ONE is allowed to skip it.

Again, good luck,

Craig

Re: Phishing testing - Consequences of failure

JK1 — Thu, 20 Aug 2020 17:17:25 GMT

Thank you all for the responses and for the resources, there is lots of food for thought to take away. I feel one thing that I didn't get across is that the thinking behind this is really that disciplinary is a last resort and we would much rather focus on awareness and training.

We are attempting to foster a culture of security that encompasses everyone from the top down and C-suite focused awareness is something I think would definitely be of benefit.

Joe

Re: Phishing testing - Consequences of failure

dcontesti — Thu, 20 Aug 2020 17:37:31 GMT

Actually one thing that I should mention:

If staff know you test quarterly, they will become accustom to it and (I like to say) become numb to them. This becomes essentially true when there are no teeth/ penalty for faiture.

I fully understand the rationale behind them but they can also have a downside.

MHOO

Re: Phishing testing - Consequences of failure

tmekelburg1 — Thu, 20 Aug 2020 18:00:55 GMT

To drive the conversation, anyone have any personal experience stories about setting up their SATE program?
Positives and negatives they experienced?
Lessons learned we can all benefit from?
You notice a considerable difference when tailoring the program to specific job roles?

Re: Phishing testing - Consequences of failure

CISOScott — Tue, 25 Aug 2020 15:12:39 GMT

Would they accept an accounting manager who "loses" some funds every year? Or an inventory clerk who has company items go "missing" every year? The problem with wiping out past history is that you lose the ability to show the risk involved in keeping the employee. I agree with others that the first steps are retraining and awareness. When those do not provide the desired result, you must have consequences.

You also have to look at this aspect: WHY did they click/open the email? Is it: 1) Ignorance (just don't know better), 2) Carelessness (know better but don't want to or care to change) 3) Willful negligence (know better but do it anyways)

If it is 1) Ignorance, then you have to see if retraining will be effective. If the person cannot be trained then you have to look at either reassignment to different job duties, restriction of rights, accepting the risk, or removal from employment.

If it is 2) Carelessness, first determine if it was an accident (wasn't paying specific attention to phishing signs) or normal behavior (doesn't ever pay specific attention to phishing signs). Once you know that, then you will have to ensure your policies have the bite in it to either wake them up or document unsatisfactory performance (which can lead to termination)

If it is 3) Willful negligence (which can be clicking on a money making endeavor like the infamous "Microsoft will pay you money for everyone you send this email to") or the person otherwise knew the risks but clicked anyways. Again you will have to ensure your policies have enough teeth to achieve the desired behavior for change. I would keep an elevated eye on this person's behavior as they introduce more risk to your agency.

I once worked at a place where we had several triggers that could get you placed on our "watch list". We had the ability to remotely connect to computers without being detected and we could watch people. We didn't just randomly watch people, we used this as our remote computer management tool. When installing software or performing repairs, we would remote into the computer. If we saw someone was working on the computer we would disconnect and try again later. There were several times where we jumped into more than what we wanted to see, and that is how some people got on the watch list. We had a separate monitor that we could see thumbnails of people's activity (too small to read information but if certain "fleshy" images popped up, we could immediately switch to full screen). Once you got on the watch list you were secretly monitored for 30 days. We also pulled Internet logs and firewall traffic and reviewed them. If nothing else was tripped, then you dropped off the watch list. We ended up firing 3 people for pornography related activity (well not really fired for pornography but for being a risk to the network by visiting websites that carried a higher risk of infections while also not being good for the company reputation). We had a good policy, ensured the security monitoring was mentioned in the logon banner (and Acceptable Use Policy that all employees had to sign) and had good evidence collection procedures to document the violations. So if you are going to be effective at discipline, you need to have good policy, ensure the people are given the chance to improve along with training and then discipline them if they fail to learn.

So find out the why of the click. That will also help you determine how you proceed. If you had multiple offenses you have to see why. Is it like most training? Extremely effective in the first few months of training and then the "awareness" wears off, OR is it that the attackers have become better and they fooled the employee who was only aware of the old tricks? So just having an arbitrary one year wipe off of past offenses can introduce more risk and could be skewing your understanding of the real risk.

Re: Phishing testing - Consequences of failure

ericgeater — Tue, 22 Sep 2020 12:12:04 GMT

I've seen where an exec clicked on a vulnerable test message, and instead of completing a ten minute retraining, he went to HR to ask politely to be removed from training. HR obliged him, btw.

Re: Phishing testing - Consequences of failure

William — Tue, 22 Sep 2020 12:27:07 GMT

Hi JK,

I'm quite surprised you'd actually think about firing someone if he fails a phishing-test a couple of times.

Let's say he or she is your CEO, would you fire your CEO if he fails your test 5 times?

Or would you fire yourself for clicking on a link to a non-company domain or something like a bit.ly link in a mail? (start counting if you would) 🙂 Or would you now dare to visit this link? https://bit.ly/isc2info?

It might be good to overthink that concept: try looking at your employees as a "human firewall" instead of "problem between keyboard and chair". If they fail a test, re-consider your awareness campaigns might not work as expected or the test is wrong (let them learn from past mistakes, don't the replacing an employee will make anything better). Give them better training, make sure they understand the importance, make them feel happy when they report a phish. You'll might even notice they start reporting other security-incidents you never heart about too if you start doing this.

IMHO It's more important that it's easy to report a phish, e.g. with a button like 'report this e-mail to our security dept." than that you think everyone will actually report an actual phishing (don't be surprised: they will not, while they might pass every test). This way actual phishing attempt will be known and actions can be taken.

Re: Phishing testing - Consequences of failure

tmekelburg1 — Tue, 22 Sep 2020 14:35:42 GMT

@William wrote:
Hi JK,

I'm quite surprised you'd actually think about firing someone if he fails a phishing-test a couple of times.

Think about the potential consequences of that person spreading ransomware or other types of malware. Depending on the industry and systems affected, it could impact human life. There's a delicate balance of when to mentor and when to fire. I'd recommend reading The Dichotomy of Leadership by Jocko Willink and Leif Babin. Specifically chapter 4: When to Mentor, When to Fire.

Let's say he or she is your CEO, would you fire your CEO if he fails your test 5 times?

No, this is a mentor situation and could probably be fixed by the CIO or CISO after the second occurrence. The Board would have to remove the CEO and it won't happen if that's the only issue.

IMHO It's more important that it's easy to report a phish, e.g. with a button like 'report this e-mail to our security dept."

That is a great idea. Some Phishing platforms have the add-in button feature available.

Re: Phishing testing - Consequences of failure

dcontesti — Tue, 22 Sep 2020 15:01:29 GMT

@ericgeater wrote:

> I've seen where an exec clicked on a vulnerable test message, and instead of completing a ten minute >retraining, he went to HR to ask politely to be removed from training. HR obliged him, btw.

I once worked in a corporation where the CEO wanted to not have a password on his accounts. Being brave (she says) or stupid, I sat down with him and explained the following:

If we let little Billy go without a password, then all the VPs, etc. will want to have their passwords removed and when that happens all the little Sallies will want their password removed and suddenly we have no passwords anywhere in the company. Guess what happens next, company financials become public before they are audited, product mixes are now public, etc.

Once he understood our logic, he became a supporter for Security and informed his direct reports that they would comply with all mandates coming from Security, including mandatory training.....

Took a bit to sit down with him, knowing I was actually risking my career but in the end paid off......of course a piece of advice, knowing the audience certainly pays.

Another trick I have used was to bring HR into the fold on Security. At one company we (I) had bi-monthly meetings with Legal, HR, Physical Security and InfoSec to discuss issues (sometimes there were also one on one meetings). This kept us all on the same page, we knew what the other was doing, thinking, etc. and there getting buy-in on programs was much easier. Legal kept me out of trouble with regulations, etc., HR kept me out of trouble with the Human Rights side of the house, Physical Security kept me posted on new tech they were implementing and where it could be used to protect the IT assets. A good blend.

regards

Re: Phishing testing - Consequences of failure

William — Tue, 22 Sep 2020 15:51:26 GMT

If you are afraid your users can share or even start malware, make sure this is technically fixed. Make sure your user can't be 'the weakest link' technically and becomes your 'human firewall' by making sure he gets alerted just-in-time and is able to get the security team involved (and compliment him when he does to create a positive feedback in your company). The worst that can happen is that someone actually installs the malware and doesn't report it as he is afraid that he gets fired.

Let’s agree that if you’re in Europe you don’t want to fire your employees for simply making a mistake that I can even let all my peers make.

Let's say he or she is your CEO, would you fire your CEO if he fails your test 5 times?
No, this is a mentor situation and could probably be fixed by the CIO or CISO after the second occurrence. The Board would have to remove the CEO and it won't happen if that's the only issue.

Please note that there is a huge difference between that you think your CIO/CISO is able to inform your CEO how to detect phishing and him not actually falling for an advanced spear phish.

If you're not convinced everyone will still be able make a dataleak even having the best training in the world, look at the news for instance where (what I could name) 'the best trainer' had an incident this year with phishing https://www.sans.org/dataincident2020 ;).

That is a great idea. Some Phishing platforms have the add-in button feature available.

https://docs.microsoft.com/en-us/microsoft-365/security/office-365-security/enable-the-report-message-add-in?view=o365-worldwide for the default exchange set-up is available, just BCC it to your phishing@company and stop the actual message to microsoft. You'll be surprised how much phishing your users are getting and never told you about 😉

Re: Phishing testing - Consequences of failure

CISOScott — Tue, 22 Sep 2020 16:16:52 GMT

@William , Yes you have to "adjust" your rules for senior leadership. I once had an ISSO who wanted to lock the CO's account for not taking the security awareness training. It was a yearly requirement. For those of you who don't know, on a military base, the CO or Commanding Officer, is like a CEO in business. My ISSO wanted to wait until his year was up and then lock his account. I told him that we were not going to do that. I instead scheduled a meeting with the CO. When it was time for the meeting I told the ISSO to come with me. When we went in I told the CO what I was there about, his failure of not taking the required security awareness training which was about 1 hour long. I said this to him: "Sir, the reason we are here is that we require everyone to complete security awareness training annually in order to remain on the network. For those who fail to do so, we lock the account and make them come to our office to unlock and take the training in our presence. You are currently overdue for the required training, however; I realize that you are very busy and have lots of stuff to do so we won't lock your account, but I was wondering if you could help us out. If you, the CO, completes their assigned security awareness training, then no one else should have a reason to claim they are too busy to complete it." He thanked me for understanding and set aside an hour at the end of that day to complete it. Then he helped us out by saying "Send me a list of everyone who is delinquent and they will get a special email from me directing them to complete it this week." So instead of using the lock out to force compliance from the CO, I used words and empathy. The CO then used the threat of lock out to help us enforce the compliance. It was a learning opportunity for the young ISSO. So treat your VIP's special and look for ways to collaborate instead of terminate. If my CIO, CEO or other executive management was failing the phishing tests I would be having lots of conversations around how important it is and how they are a target and they have to be more careful. I would discuss the ramifications of compromise of their account versus a "regular" user.

Re: Phishing testing - Consequences of failure

William — Tue, 22 Sep 2020 18:15:32 GMT

Beautiful example. Very good also that you only lock the account, not fire the user.

Though, think what would happen if your CO would do the same that was done for him with every 'delinquent' (man, what a word for a colleague that didn't do a test ;)). I bet the employees would be so surprised they talk about it on the floor how good he is and how seriously he takes those tests. Just a mail and if not followed up within a week that 2 minute talk from a CO.

And just reserve the locking for those users that knowingly keep rejecting and don't provide a valid reason (these are the colleagues you might want to go work with a competitor :)).

If you're doubting: think about a scenario where a medical is saving patients lives every day. He is very good at it, but doesn't understand computers the way you, me and the C(E)O does. Now, if you lock his computer, that patient might die as he is missing vital info. You definitely want him to learn about phishing, obviously. Firing a good doctor will cause issues obviously, locking his account too. Just talking to him, telling him to free up an our next week (and making sure he can do that) is beneficial for the whole of the organisation.

p.s. If your awareness campaign takes an hour, you definitely have to much content inside. People will never remember what you'll be saying. Make it 10 minutes, and predictable (like every month on monday at 08h00 they have the mail and they can do it whenever they want within that month, and they are free to skip it 3 times before someone complains). And having the campaign in such a way people like it (eg. they feel rewarded, for instance by something simple as winning free webcam stickers, t-shirts for the first X that finish etc.etc. ) they will do it anyway. If you need an example, look at ' https://www.certifiedsecure.com/certification/view/45 and hit the 'don't click plus' training'. Many of the colleagues I speak to are addicted due to the game-element in that training.

Re: Phishing testing - Consequences of failure

tmekelburg1 — Tue, 22 Sep 2020 18:15:20 GMT

@William wrote:

Please note that there is a huge difference between that you think your CIO/CISO is able to inform your CEO how to detect phishing and him not actually falling for an advanced spear phish. If you're not convinced everyone will still be able make a dataleak even having the best training in the world, look at the news for instance where (what I could name) 'the best trainer' had an incident this year with phishing https://www.sans.org/dataincident2020 ;).

Sorry for the confusion, this is not a mentor lesson on how to detect phishing attacks. This is a discussion centered on the SATE program itself, the importance, and any feedback from the CEO on how they believe it could be improved for better effectiveness. CISOScott's post elegantly laid it out on how that meeting can go as well.

I think for us to agree we would first need to figure out when a mistake is no longer considered a mistake and would be considered negligence. If we use the OP's limit of five, the fifth "click" at this point is no longer a mistake in my eyes.

There could be other factors involved here as well. For example, initially when we first started the phishing campaigns they were easy to spot. When everyone went through training, we then set the difficulty higher. So obviously there were more people who clicked and we didn't make a big fuss about it because that's what we expected. We used it as 'teachable moments' and now they are afraid to click on anything (insert evil laugh).