Addressing false positives in automatic scanning

By BSIMM Editorial Team posted 12-04-2020 01:39:00 PM


In years past, traditional development practices were slow enough to ensure adequate time for manual review and triage activities within the development process. But with the ever-increasing speed of DevSecOps today, this is no longer possible—quicker, continuous deployment demands more effort in less time. Agile development processes require greater velocity; releases that may have been quarterly are now required weekly or even daily. Development is tasked with manual triage—an approach often with no strategy or consistency, and that inevitably causes an overload of false positive results, and ultimately, distrust in automated scanning itself.

If you’ve ever used an automated security testing tool, you’re no stranger to false positives. Pesky false positives “lie” to you and demand time and manual effort to sort out. And they pull your valuable resources and time away from more important and impactful work.

Beyond the burden on security, key business strategies and initiatives may also suffer as a result of these diminished resources. This is especially true in the case of a sophisticated automated build and deployment process (your CI/CD pipeline), where false positives pose an even bigger roadblock, threatening to delay and stall your deployments and negatively impact your application security and overall business results. 

Understand the critical impact of false positives

False positives can have a negative impact in a variety of ways, and on many stakeholders.

Development team. With faster and continuous deployment, more effort and resources are needed to address false positives. Developers are pulled into laborious manual reviews, providing less availability for development and innovation.

Security. False positives directly translate to decreased application security. The more developers and security teams distrust their automated scanner, the more they will ignore and dismiss vulnerabilities identified by the tool. Furthering the risk, an overload of false positives makes it more likely that a real vulnerability could sneak by unmitigated.

Team dynamics. Whenever an unmanaged automated tool reports a vulnerability, developers must investigate and remediate the issue. The more false alarms are reported, the less interteam trust exists, resulting in a deteriorating relationship between development and security teams.

Financial. Resources of all types—time, people, tools—are required to verify false positives. This results in unnecessary costs across development and security teams. One critical area of wasted spend is security tooling licenses: when development ignores tooling and security fails to improve the process, the business is squandering its license spend.

Combat the time-suck and expense of false positives

In the effort to minimize the negative impact of false positives, it’s important to first reiterate the fact that your automated tools are just that; nonhuman tools, with no intelligence. Your automated scanners don’t have any context when performing tests, meaning they have no way of knowing whether a finding is truly a bug, or whether it’s an error (a false positive). 

Given this, your approach to minimizing the impact of false positives should be viewed through the lens of creating context for your scanners and structuring your scanning practices to provide the most optimal results.

Let’s dive into some key areas you can focus on to reduce the impact of false positives.

Manage the signal-to-noise ratio

The “noise” from security scanning tools, when unmanaged, leads to a cacophony of input capable of overshadowing real and concerning security threats. The problem with scanners is not the data that they produce, but the management of that data. When your signal-to-noise ratio is too low (there’s too much unmanaged data), your teams will miss valid and important data.

The only way to tackle this data flow and avoid overlooking important information is to fine-tune tools to collect the correct data at the correct time. This will be different for each organization, but the guiding factor should be to identify critical assets and the indicators of attack on those assets. Once these prioritized indicators have been identified, tools can be programmed to filter out unnecessary input and focus primarily on these key indicators. 

Create a strategic triage plan

Without a strategic plan that spreads the responsibilities of triage across individuals and teams, developers often become too overloaded to perform their job adequately.

Sometimes development teams are even responsible for scanning the entirety of their development pipeline. The sheer scale of this task leads developers to ignore the noise of static application security testing / interactive application security testing tools, resulting in a failed security plan.

Efforts should be spread across teams so that responsibility overload doesn’t result in entire security practices being overlooked.

Create a baseline

Security teams should help the development team when setting up and calibrating automated tooling. It’s critical to perform some baseline scanning before turning the development team loose. By creating a baseline, overall system noise is reduced.

Setting up a baseline involves someone reviewing an initial scan, marking false positives as false, and creating rules and filters to tune the tool for future scans. This individual also configures the development in such a way that the baselines are merged with future scan results in order to remove any old false positives. 

Creating custom rules is imperative, as it helps to filter out noise and fine-tune. By constructing a customized baseline, the development team can more effectively manage all scanning activities and results.

Enforce security team support

It is not realistic or wise to expect a development team to be able to single-handedly manage automated software security scanning. Setting expectations that the security team or a designated “security champion” will be available to help with issues as they arise helps set teams up for success and makes the process sustainable.

Set expectations for roles and responsibilities

Without an adequate staff of employees responsible for addressing false positives, no processes are maintainable. Be sure that you have enough of the right people and that they clearly understand their role and responsibilities.

Fostering and identifying security champions, wherever possible, is very important. Thought should be given to who is responsible for identifying these champions and encouraging their relationships and outreach.

Define policy standards

Set up precise rules and processes for developers and security teams so they don’t get bogged down in the unknown. For example, after x number of false positives, security will get involved.

Create rules and expectations that keep processes and responsibilities balanced and help alleviate any confusion or gray areas. The clearer the expectations, the smoother the process.

When creating these policies, you should be asking questions like:

  • Who verifies false positives?
  • What are the expectations of the dev and security teams?
  • What evidence is required from the dev team?
  • Who defines and enforces these policies?
  • What is the escalation process?
  • What is the acceptable ratio of false positives on subsequent scans?
  • When the acceptable ratio of false positives is exceeded, what actions occur to resolve the issue?

Minimize the impact

Although false positives can throw a wrench in your development processes, setting up adequate policies, practices, and procedures can help eliminate their negative impact.

Clarity around and communication of expectations, roles, and responsibilities, as well as practices targeted at minimizing the impact of false positives can reduce the burden of automated testing.

Set your team up to succeed by implementing a process to support automated scanning and providing ongoing guidance for integration and automation.