How to Avoid Getting Caught in the Statistical Compliance Testing Trap

Using highly manual security assessment processes—and legacy tools—makes it much more likely that you’re going to overlook the hidden parts of your customers’ attack surface.

Security Compliance

One of the most commonly repeated mantras in channel security circles is that global system integrators (GSIs) and managed security service providers (MSSPs) and their customers should adopt “multilayered security.” The problem with this advice is that after observing several companies that suffered a cybersecurity attack or breach, they were using multiple security solutions and services.

For example, the deployment of vulnerability scanners, penetration testing, threat intelligence feeds, security ratings services and other tools and services are routinely used. In fact, security frameworks and compliance regimes, such as the Payment Card Industry Data Security Standard (PCI DSS), specify the systematic use of these tools. Yet, despite adherence to these requirements, we frequently read about successful exploits. Why is that?

Key reasons cybersecurity strategies miss the mark

One of the biggest security mistakes is underestimating the customers’ attack surface. First, let’s define this term. An attack surface includes any asset that an attacker may see that provides a path to your customers’ network. A midsize company’s externally exposed attack surface can consist of hundreds of networks, thousands of devices, hundreds of applications and dozens of connected partners.

A recent ESG study commissioned by CyCognito offers insights into how companies are doing protecting these attack vectors:

  • 47% don’t include SaaS applications in their vulnerability scans
  • 45% don’t monitor workloads running in the public cloud
  • 45% don’t scan third parties connected to their networks.

Virtually all security tools are only good at testing the targets they’re programmed to track. Consider how a typical vulnerability scan works: the admin enters a target range of IP addresses, and that’s where the tool looks. If you want it to look somewhere else, you must tell it where to execute its processes, what to look for and when to run scans. As a result, vulnerability scans are looking at only a fraction of the attack surface. For penetration tests, the focus is on an even smaller subset of assets, creeping for hours through a “to do” list that barely scratches the surface of potential exploits.

Auditing requirements, such as those included in the Payment Card Industry Data Security Standard (PCI DSS), can exacerbate the problems described above by giving admins a false sense of security. For example, to achieve PCI DSS compliance, admins merely have to test a subset of their data (i.e., a sample) to satisfy auditors that controls are implemented as expected. Unfortunately, it’s become a widespread practice among security managers and even some GSIs and MSSPs to equate “compliance” with “best practices.” Don’t fall into this trap.

There’s another pitfall associated with sampling: it assumes that a security team has a clear view of all the assets in the extended attack surface. Additionally, the frequency of testing often leaves much to be desired. For example, research from Informa Tech shows that 72% of organizations pen-test quarterly or less frequently and only 22% test monthly or more frequently. 

Why automation is no longer an option

Given that testing for complex attack vectors can be cost-prohibitive and time-consuming when using highly manual processes, it’s not surprising that so many organizations take shortcuts and settle for compliance. But, after considering the increasing frequency and costs of security incidents and breaches, it’s obvious this isn’t a viable strategy.

Attackers are working overtime to identify the hard-to-find vulnerabilities in your customers’ networks, devices and apps. Whether there are accidentally exposed sensitive data, authentication and encryption weaknesses, misconfigured applications or assets, a network architecture flaw or some other vulnerability, it will be found. The question is will it be your company that finds it first or a cybercriminal? If you’re relying on highly manual tools and processes, the odds are tipped in favor of the bad guys. But, on the other hand, if you’re taking advantage of automated tools that make it more practical and cost-effective to address 100% of your customers’ attack surface, you—and your customers—will have the upper hand.


Randy Streu

Randy Streu is Vice President Business & Corporate Development for CyCognito, a company focused on solving one of the most fundamental business problems in cybersecurity: seeing how attackers view your organization, where they are most likely to break in, what systems and assets are at risk and how you can eliminate the exposure.