This article originally appeared on CSO Online as part of the CISO 2.0 blog series. You can read part 1 and part 2 of the series in the links provided.
In the first and second parts of our series on engaging with boards on cyber risk management, we explained what today’s boards needed to know about security. In this article, we’ll deep dive into some of the metrics associated with our four-step methodology for qualifying threats and prioritizing risk (see details in part 2). The ultimate goal of metrics and controls – and of the entire security organization – is to lower risk to a palatable level for the business.
These metrics will allow you to report risks to the business in ways that will resonate with boards. As we discussed in part 2, our industry often relies on numbers like “300,000 malware alerts” to explain risks. These quantitative terms offer little context for the listener (It’s not that we should never use numbers, but perhaps those numbers are best used only within circles where they are guaranteed to be understood.) We also often use qualitative statements, like “medium risk of a cyber attack,” that, in isolation, are too vague to be useful.
A refresher on metrics
When you’re presenting such data to people outside the security team, it’s important to tie specific risks to specific information assets. You’ve now offered numbers that mean something to non-security people. In this way, you demystify the jargon and esotericism associated with security. Start by asking:
- What’s important to the organization or customers?
- Who, within the organization, is responsible for safeguarding the important “stuff”?
- What could the bad guys do to us if they gained access to that stuff?
- How do we stop or mitigate what the bad guys are doing?
Quantitative metrics, which are great for the techies, are based on actual figures: You can place a dollar value on quantitative metrics, or you can assess an impact or likelihood in numerical terms. Quantitative assessment leaves less room for error, but such measurements are not always possible. This is where qualitative metrics come into play, provided by cyber and information security experts. These metrics give our business leaders advice and guidance, which helps them run the business better.
Understanding KRIs and KPIs
When we engage with the board, we should split reporting on metrics into two categories:
Key Risk Indicators (KRIs): Think of KRIs as warning signs – indicators for the board that there could be security challenges ahead which affect the achievement of business objectives. KRIs should inform your forward-planning. KRIs will identify how and where future investments should be made: by looking at the threat landscape, and understanding who might be attacking the business. By providing KRIs, security teams can help the business make the following decisions about risks:
- Accept: The risk is within the organization’s “risk tolerance.”
- Mitigate: The organization can apply controls to lessen the impact or likelihood of the risk to a more palatable level.
- Transfer: The organization can pass the liability to another party. Cyber insurance is a good example of transferring risk
- Ignore: This is very different from “accept,” which is a conscious, pragmatic business decision! No organization can transfer or mitigate all of their risks but where a risk is accepted, the decision must be made through a balanced risk assessment.
The ISF IRAM2 is an excellent source of information for articulating the types of threat actor that might be interested in your enterprise; its methodology is an excellent way for you to classify data in business-aligned, jargon-free terms. NIST 800-30 also provides a practical approach to risk management and follows a similar set of conceptual steps. It’s sequential, pragmatic, and relatively easy to pick up.
For KRIs to be meaningful, we need to better understand what we are looking to protect. This is where information security and business continuity functions must work together. We must create Business impact Assessments (BIAs) to understand and prioritize precisely what the company cares about. Check out the ISO 22301 standard, which covers the components of a business continuity framework. As part of your BIA framework, make sure you’re classifying the value of your information. Keep in mind that we cannot protect all data in all locations with the same set of security controls. That’s expensive and could unnecessarily impact productivity.
When considering cybersecurity risk, it’s important to ask: “who cares”? The security function is there to apply safeguards which protect the assets a business cares about. How do we find out what is important? We get to know the business. A process that has served me well is reading the company business strategy and identifying the things that matter. I have included some examples below:
Key Performance Indicators (KPIs): If KRIs are looking forward, KPIs are there to validate existing investments in security – what you’ve already implemented. This is how you describe the efficacy of your solutions.
There is an intrinsic link between KRIs and KPIs. It’s easy to fall into the trap of having many KPIs without the associated KRIs – this is how we end up offering metrics like “17 failed login attempts for SRV001” without any context. What is housed on SRV001? Where are the connections coming from? Lots of data, not a great deal of information. KPIs can include uptime of servers, patched systems, the state of compliance (such as PCI-DSS percentages), and SLAs associated with security changes. (But this is by no means an inclusive list).
The Importance of Trending
Executives don’t want numbers without context. They want a picture of how their environment is performing, and information about any impendent to the business realizing its goals. When we report, we should provide indicators of improvement. I’ve worked with arrows, heat maps/thermometers. There’s nothing wrong with H/M/L as long as you’re doing the sums in InfoSec which show how you got there!
Remember that business teams define the importance of information. The security team applies controls commensurate with that classification. Once the business tells us what’s important, we can then go about applying safeguards.
If you are looking for a seat at the board table, you need to inspire and motivate your senior executives. Always look to present information which shows areas of improvement across a defined period of time, trending. Are botnet calls reducing quarter-on-quarter? Is the consumption of unapproved cloud applications on the rise? Show meaningful information and explain the impact this is having on the business.
In the next article – the last in our metrics series – you’ll learn how to create metrics based on the KPIs and KRIs that you have generated.
