What is the best method of testing the effectiveness of a security awareness training program?

Phishing attacks, stolen credentials, business email compromises and other threats that take advantage of human error continue to trouble businesses. Which is why it is absolutely vital that your cyber security awareness program is effective, AND that you are consistently measuring it for its efficacy. While cyber security awareness programs can have a variety of goals, typically they are designed to educate employees so that you can protect the company.  However, without employee buy-in, your cyber security awareness program may not be totally effective. 


How Effective Is Your Cyber Security Awareness Program?

Download this quick guide to measure the effectiveness of your cyber security awareness program beyond just the compliance reports.

What is the best method of testing the effectiveness of a security awareness training program?


As a Security and Risk Management Leader,  it is imperative that you test your cyber security awareness program in order to validate that your initiatives are effective. Here are 10 indicators that show your employees’ awareness is improving.

1. How Many People Reported Phishing, Loss Devices, or Other Incidents?

When security awareness is going up, you'd expect to see an increase (at least initially) in the accurate number of reports coming into InfoSec.

2. Is There a Decrease in the Amount of Clicks From Phishing Tests?

Phish testing puts the effectiveness of security awareness training to the test by reinforcing what has been presented. Results of the testing are evidence of effectiveness.

3. Is There a Decline in the Amount of Confirmed Incidents?

When your cyber security awareness training is effective, you would expect to see an overall decline in the amount of incidents year over year.

4. Are the Number of Policy Violations Going Down?

Adhering to security policies shows maturity in the security culture. It is usually a result of understanding why we implement these controls and an open door to the security team. Instead of bypassing these controls, people feel comfortable reaching out to the security team.

5. Do Employees Ask Questions?

A great way to measure engagement is to track how often employees ask questions. This could be through a ticketing system, google forms, or in-person.

6. Is the Security Team Involved in More Projects?

Measure how often people are asking the security team for help to ensure their projects are “secure by design.” 

7. How Many Requests for New Technologies?

Prior to security awareness training, people may have used unauthorized apps to bypass security controls - commonly referred to as "Shadow IT." If people are now asking for permission to use new technologies, it is a sign they understand the risk and wish to mitigate it. This also shows healthy collaboration with the security team where people are not afraid to ask for assistance.

8. Are People Participating in Non-Mandatory Training?

When people proactively consume your content it is a great indicator they are interested and engaged. So offer optional training like "Online Family Safety” or lunch and learn sessions, and track how many people signed up or took the training.

9. How Deep Do They Go?

If you have analytic tools you can measure how deep people dig into your content, similar to how it’s done with your website. For example, how many pages did they view, how much time did they spend consuming your content, etc. The more content they consume the more engaged they are, but this also requires high quality content.

10. Observe Behavior

Similar to how we observe our kid’s behavior when displaying respect for others, we can also do the same simply by walking around the office. You can observe people's behavior, for example how often sensitive information is laying around or do people still use sticky notes with their passwords. Some examples are do people check badges of others they don’t know, has tailgating increased, are assets left unsecured, or are doors closing completely.

How is security awareness measured?

Program effectiveness can be measured by capturing data on changes in the way people react to threats, such as the ability to recognize and avoid social engineering attacks. It's important to objectively monitor the performance and impact of an awareness campaign using data and metric-based tracking.

What is the most important factor to consider when designing an effective IT security awareness program?

Most important of all, however, is being able to show proof that the changes being put in place are making an impact. Having data to show where you were before versus where you are after implementation is required to prove that the organization has not wasted time and money.

What is the most important security awareness training?

Organizations looking to heighten security awareness among employees need to cover a wide variety of security awareness training topics, but social engineering tops the list.

What makes a cybersecurity awareness training effective?

Other Considerations for Effective Training Cyber awareness training is about educating and changing employee behavior enough that it increases your staff's ability to consciously make more secure decisions in your environment. That is much more easily said than done. Making the message relevant is key.