Vulnerability Assessment Data Analysis

Ed Bellis    August 29, 2011

A couple of weeks ago we asked you to help us build our roadmap by completing a short survey. We wanted to ensure we were integrating with the vulnerability assessment and remediation tools that were most valuable to our audience. I initially thought we would be able to pull this information from the analysts but had very little luck with any meaningful information. The reports I received didn’t align at all with what we were seeing in the field. So we thought, what the heck, let’s just ask!

To incent participation we promised two things.

1. One respondent would receive a free one-year enterprise subscription to our Risk I/O vulnerability management service.

2. We will publish the data.

So without further ado, we’re fulfilling on both. First I wanted to congratulate Jeremie Kass on winning the enterprise subscription. And thanks to all that filled out the survey. The data we were able to collect was far more valuable than any of the reports we previously had.

We had a total of 47 respondents last week. While not a massive amount of data, we felt it was a large enough sample size to provide us with a good indication of what’s being used. Before we jump into the tools, lets take a quick look at the demographics by company size.

What is the approximate number of employees within your organization?

As you can see, our respondents tend to fall in either smallest or largest categories, with a minority in the middle. I suspect this will shape the results a bit but let’s take a look. First up the dynamic analysis tools.

What vulnerability assessment tools do you currently utilize within your organization?

As you can see, outside of maybe Nessus there was no clear dominant dynamic scanning tool. The distribution remained fairly even throughout all the way through the other category. It’s important to note these graphs only include the top results. At the end of this post is a link to the raw data in it’s entirety if you’d care to review.

Next up we asked about the static and binary analysis tools you were using.

What static code and binary analysis tools do you currently utilize within your organization?

Wow, one thing here obviously stands out. The majority of respondents aren’t using these! My initial suspicion is since we had many respondents in the ‘small business’ category it was skewing these results. We’ll pull the enterprise respondents separately later in this post and attempt to validate that.

Finally… Which, if any, trouble ticketing and bug tracking systems do you use within your organization?

None? really? That was a bit of a surprise to me. I thought perhaps this had something to do with the industry demographics. Perhaps our respondents were largely from consulting and professional services where assessment was their responsibility but not remediation. But that wasn’t the case. According to the data, only 2 respondents were in professional services or consulting.

So let’s go back and take a second look at the Static & Binary Analysis tools through the lens of the enterprise. If we filter those results to only companies with more than 10,000 employees it looks like this:

Turns out to be very consistent with the larger pool.

A couple of additional data points worth mentioning:

  • 77% of respondents used more than one dynamic vulnerability assessment tool
  • 17% of respondents used more than one static or dynamic analysis tool

Of course it goes without saying we need to state a couple of obvious facts here. First, this is a pretty limited sample set (47 respondents). Also, there is some selection and audience bias here. If you follow us or some of the other folks who served as referrers to the survey, you tend to be further along in your vulnerability assessment and management and all of this should be taken into account.

As an added bonus for the data heads in the crowd, we ran a very similar survey throughout our closed beta and received about 60 entries. I am including links to the raw data in csv format for the current survey and the previous beta survey. I’d love to see what additional interesting views may come from this or have someone expand on it. Of course we are always happy to accept more data if you’d like to take the survey.

Leave a Reply

Your email address will not be published. Required fields are marked *