Author Archives: Tony Bradley

Metrics are an important element of making effective business decisions. When it comes to security, metrics can help you determine the performance of current security tools and processes, and identify weaknesses or areas to be improved. Security metrics can also help you identify and thwart an ongoing attack against your network or data. That assumes, however, that you’re looking at the right metrics and acting on the information appropriately.

Wrong metrics yield wrong results

Consider the Titanic, and let’s assume for a minute that metric data was being collected and reported, ostensibly to ensure the ship safely navigates through a sea of icebergs. How valuable would it be for the captain to receive a report detailing the number of deck chairs on the ship, along with how many of them were damaged and in need of repair? Zero.

Capture and analyze the right information

Organizations need to have tools and processes in place that enable them to capture and analyze the right information. From a security perspective, it helps to work backward. Consider what a successful attack looks like, and the events and activities that lead up to it. That way you can identify the appropriate indicators of compromise that should trigger an immediate response.

Right metrics, wrong process

Let’s go back to our Titanic example. It would obviously have been much more valuable for the captain of the Titanic to receive a report detailing the icebergs that had been identified in the path of the ship, along with a series of recommendations for how to adjust navigation to avoid them. If the captain did not regularly view the metrics reports though, and if there was no process in place to separate important information about icebergs from irrelevant information like the number of deck chairs, the results would be the same.

Separate important information from irrelevant information

Another example of having the right information with the wrong process was the data breach of U.S. retail chain Target in late 2013. A 2014 article explains, “Target confirmed Friday that the hack attack against the retailer’s point-of-sale (POS) systems that began in late November triggered alarms, which its information security team evaluated and chose to ignore.”

Differentiate critical alerts from trivial alerts

In other words, Target had the right security in place, and the tools to generate the alerts necessary to make security personnel aware that a critical event was happening, but the process for differentiating critical alerts from trivial alerts and responding appropriately to that information was flawed.

Doing metrics the right way

The Titanic examples illustrate that there is a right way and a wrong way to do metrics. First, you have to be focusing on the right metrics—gathering data that actually matters for the important decisions you need to make. Second, you must be able to separate the signal from the noise and have a process in place to ensure that critical and/or timely information is seen and acted upon appropriately.

Respond appropriately

Not all metrics or the processes for handling them need to be about ongoing attacks or urgent incident response. If the iceberg threat was addressed, the captain of the Titanic might still be interested in the current state of deck chair repairs. Gather and report data on as much as you can. Just make sure you can differentiate trivial data from important data—that you can separate actionable intelligence from general information —and ensure that you have the processes in place to respond appropriately to the metrics that matter most.

More information

Recently, Tenable sponsored publication of the ebook, Using Security Metrics to Drive Action: 33 Experts Share How to Communicate Security Program Effectiveness to business Executives and the Board. The ebook is a compilation of essays by security officers who share their best practices for implementing an effective security metrics program. Download your free copy for a gold mine of advice.

It’s that time of year. Once we’ve run out of leftover Thanksgiving turkey for sandwiches, and our Black Friday purchases start to show up on our credit card statements, there are two things that seem to happen every year: reviewing the year gone by and making predictions for the year to come. I get it. It’s tradition. However, few ever learn any lessons of value from analyzing the events of the past year and even fewer gain any relevant insight into the year ahead from speculative prognostications—especially because most are either safe and obvious predictions or end up being wrong anyway. With that in mind, I have created a new list of reasons to stop doing that.

1. The past may not be relevant

Lots of stuff happened in the past 12 months. Even if we narrow the focus just to network and data security and security incidents, there’s no shortage of events to reflect on from 2016. However, most of those events affect platforms or technologies you don’t use, or target industries you’re not in, so reviewing them provides little value aside from increasing your knowledge of general information security trivia.

2. The past is not an indication of the future

The events that do relate directly to your industry or company provide greater value, but knowing what happened last year isn’t necessarily helpful for preventing future attacks or breaches. Reacting to past attacks leads to things like taking off shoes at TSA checkpoints. It might have been an effective means of preventing a past attack, but has little—if any—actual impact on preventing future attacks. It’s like closing the barn door after the horses have escaped.

3. Most predictions are wrong

Making predictions about technology or security is a bit like predicting the weather, and has as bad or worse odds of being accurate. Predictions are either painfully obvious—in which case they don’t provide any insight or value at all—or tend to be guesses more than predictions. The guesses are hopefully backed by some intelligent analysis of past and current trends or knowledge of cutting edge technologies, but ultimately they tend to be wish lists more than predictions, and those making them are as surprised as anyone else if or when they come true.

4. Predictions may not be relevant

There are some predictions that end up being accurate. A combination of “even a broken clock is right twice a day” and throwing enough ideas out there results in at least something eventually coming true. Odds are that the few predictions that prove to be accurate will fall into the category of things that don’t apply to your industry or to the platforms and applications your business relies on.

5. Better to look within

Make your own lists. There’s nothing inherently wrong with reviewing the past year or trying to make some educated guesses about the year to come. It makes good business sense. However, rather than relying on technology pundits to provide you with cookie cutter “Top 10” lists, you should analyze your own data and your own history, and use the information you have available to make your own predictions for 2017—predictions that are directly relevant to you and your company.

Analyze your own data and history to make your own predictions for 2017

If you simply must read the post mortem reviews of 2016, or check out the predictions and guesstimations for what 2017 has in store, feel free. I mean, it is tradition—like green beer on St. Patrick’s Day, fireworks on the 4^th of July, or the Detroit Lions playing on Thanksgiving Day. Just do so with a realistic understanding of the value—or lack thereof—those provide for your business, and focus on the things that will actually make the most impact for you.