Don’t Let Compliance Lull You Into a False Sense of Security
Written by CISOSHARE
January 31, 2019
25 min read
Progress-based security program development focuses on the ability to make good decisions while being able to implement those decisions in the shortest possible time frame.
Unfortunately, most organizations aren’t taking a progress-based approach to security. Instead, most are strictly complying with different best practice frameworks such as ISO 27001 or NIST 800-53, suggestions from certification and cyber scoring companies. The problem with relying only on these frameworks is that they all have fatal flaws that work directly against making effective progress in developing a security program.
13 Flaws in Security Best Practice Frameworks:
1. There’s no analytical proof that compliance with best practices reduces the probability of an organization being breached by a cyber attack.
2. Auditing firms that grant certifications want you to pass so that you contract them for your audit next year.
3. There’s no regular scope on auditing for compliance frameworks, meaning that your program can be incredibly high-leveled or too specific to be effective for your entire organization.
4. The requirements listed in the frameworks are often only vaguely defined, leading to program components that are inefficient or entirely ineffective.
5. Best practice frameworks focus only on compliance, rather than the individual goals and objectives of an organization.
6. They only require that processes be documented, rather than measuring the maturity or outcomes of these processes.
7. There’s no consideration for the time that these processes take and how they help reduce vulnerabilities in a timely manner.
8. Frameworks don’t consider the amount of resources it would take to properly and consistently conduct the processes that they require.
9. Most frameworks are developed in isolation of each other and rely on secondary services to identify correlations between them.
10. Frameworks don’t have any requirements for communication systems in security, despite its importance for informed decision-making.
11. Their recommended risk-based approach ends up being entirely ineffective because of the way the frameworks are structured.
12. You can develop different components of your security program and have them all be compliant, but this doesn’t mean you’ll have a cohesive system.
13. Best practice frameworks create a false sense of security for compliant organizations.
I have to say it, and I’m sorry if it makes the information security world look bad, but it must be said:
If your organization is using strict alignment to existing best practice frameworks in 2019, your organization is going to fail in the current security landscape.
Here are the 13 reasons why:
1. Lack of Scientific Approach in Best Practice Framework Development
There’s no analytical research that correlates the development of any best practice frameworks and their ability to reduce the probability of a cyber attack on an organization.
Best practice frameworks are often developed with vendors and seasoned security professionals working together to create it. It isn’t effective, and it shouldn’t be the way that frameworks are built, but that’s how it’s done.
2. Bias in Framework Certification Assessments
If your organization wants a certification with ISO 27001 or other frameworks like HiTrust, you pay an audit firm to conduct the review. These auditing firms are invested in having organizations pass the certification to improve the number certified companies.
These audit firms aren’t likely to give you a long list of items that need to be remediated because they want you to be happy with the results in the hopes that you have them audit you again next year.
Another thing to note is that I believe that it’s important to have the people who measure your program also support you in improving it. Most best practices suggest having a separate team or group work on remediating your environment, but the group that identifies these areas for improvement might be the best suited to helping you remediate it. (More on this later.)
3. Irregular Scoping Control in Frameworks and Assessments
In ISO 27001, the organization seeking certification is the one that establishes their scope at the very beginning of the process. This means that your organization can scope out whole areas of your business or make the scope so high leveled that the resulting security program doesn’t end up being effective.
This is one of the primary flaws in the newest diet pill strategy in security: automated cyber security scores.
Scores from these cyber security rating systems often use limited scope inputs that can be attained quickly, such as publicly available technical information about an organization. This in turn impacts the score without accounting for aspects of their security program. As limited as these scoring systems are, they can at least provide a limited value to companies quickly.
4. Lack of Definition in Requirements
All the available security frameworks focus on presenting requirements that an organization must comply with. However, the definition of these requirements is often vague, meaning they can’t support the development of a valid security program that can meet a defined objective.
For example, there are requirements that require a risk register, or a risk management program, or an incident management process. But given the way that these requirements are written, an organization can comply with these requirements without having functional security program aspects.
Simply complying with the requirements doesn’t mean that you’ll have a true measurement of risk, the ability to support an incident, or the ability to make business decisions based on your security program objectives.
5. Focusing on Compliance Instead of Objectives
If you’re strictly complying with a best practice framework or security certification, that becomes the objective of your entire security program system — it becomes a full-time job. But for most organizations, this isn’t the overall goal you want your security program to meet.
More applicable objectives are to meet specific customer requirements for cyber security, or actively try and prevent a breach. Trying to meet all these objectives will lead to resource competition in your security program, which is difficult in a security landscape that is already running short on resources.
6. Lack of Attention to Process Maturity
Most security frameworks only require that a process be documented. It doesn’t consider what inputs drive the process, any tool integration, resourcing, or frequency of process performance. It also doesn’t consider how these items affect the outputs of your processes, essentially rendering the strict definitions of these requirements useless.
7. Consideration of Time Implications
Time is one of the most important aspects of reducing the likelihood of threats. The longer a vulnerability exists in your environment, the bigger the threat it is to your environment.
Best practice frameworks place limited, if any, emphasis on reducing how long vulnerabilities exist in an environment through efficient process design, technical safeguards, or any other means. Most frameworks simply ask that a program have a safeguard such as a firewall, with no correlation to how this relates to reducing the time a vulnerability exists.
8. Lack of Attention to Resourcing
Best practice frameworks don’t keep resources efficiency in mind. The focus is purely on compliance with framework requirements.
If you try to align strictly to a framework like NIST 800-53, it would be entirely inefficient to staff. The framework itself provides no guidance on estimating the true resource requirements. If the framework did this, they’d likely be showing an implementation that’s impossible for the common organization.
9. Frameworks Developed in Isolation
Best practice frameworks are often developed in isolation of other frameworks and usually depend on a secondary service such as the Unified Compliance Framework, or HiTrust to correlate the relationships between them.
The problem again is that these secondary systems still don’t focus on efficiency, only compliance. Although they may correlate the different relationships, this still leads to inefficiency.
10. Lack of Attention to Communication Systems in Security
One of the most important components to a healthy security program is a communication system. How information travels into a security program, out of it, and in between the people who manage the program is vital to operating and making informed decisions.
Best practice frameworks don’t have any requirements on communication mechanisms at all.
11. Focus on Risk-Based Approach
All the frameworks move to guide organizations to use a risk-based approach as the core component of alignment. In theory, focusing on risk is great as it should move away from the prescriptive compliance-based approach of following a simple checklist.
However, because of all the inherent limitations of a framework — scoping, timing, definitions, resourcing, maturity — result in ineffective risk-based approaches. The risk approaches in these frameworks may be compliant, but they hold very little functionality in helping organizations measure and manage risk.
12. Measurement in Parts Instead of a System
When an organization is strictly complying with a best practice framework, they can define all the different parts of their security program without building an efficient system for how these components work together.
For example, your organization may have a compliant risk management program and a compliant incident management process without these parts working together efficiently. The data from these two components may never reach the other, or any of the other parts of the program.
It’s like building different parts of a car without making sure that the car drives well or is suitable for the type of goal you’re trying to reach. Maybe you’re trying to win a race, but the framework you’ve picked has been helping you build a mini-van the whole time.
13. Creating a False Sense of Security
This may be the biggest flaw in all the best practice frameworks or certification models in cyber security. I’ve participated in so many board or leadership meetings in organizations recently that have recently been breached despite their ISO or HiTrust certification, and most boards don’t understand what happened.
It’s because of all these flaws in the frameworks that we’re using in the information security discipline. They think that these frameworks mean something, but they’re being deceived. Of course they’re upset, and it’s time for the industry to move past this.
Conclusion
Before we move on, let me be clear in saying that there is value in best practice and compliance frameworks. They provide valuable starting points in a progress-based approach to security, but companies can’t continue to rely on these frameworks by themselves anymore.
In the next article, we’ll look at what a progress-based approach to security program development means.