toolsmith #128 – DFIR Redefined: Deeper Functionality for Investigators with R – Part 1
“To competently perform rectifying security service, two critical incident response elements are necessary: information and organization.” ~ Robert E. Davis
I’ve been presenting DFIR Redefined: Deeper Functionality for Investigators with R across the country at various conference venues and thought it would helpful to provide details for readers.
The basic premise?
Incident responders and investigators need all the help they can get.
Let me lay just a few statistics on you, from Secure360.org’s The Challenges of Incident Response, Nov 2016. Per their respondents in a survey of security professionals:
- 38% reported an increase in the number of hours devoted to incident response
- 42% reported an increase in the volume of incident response data collected
- 39% indicated an increase in the volume of security alerts
In short, according to Nathan Burke, “It’s just not mathematically possible for companies to hire a large enough staff to investigate tens of thousands of alerts per month, nor would it make sense.”
The 2017 SANS Incident Response Survey, compiled by Matt Bromiley in June, reminds us that “2016 brought unprecedented events that impacted the cyber security industry, including a myriad of events that raised issues with multiple nation-state attackers, a tumultuous election and numerous government investigations.” Further, “seemingly continuous leaks and data dumps brought new concerns about malware, privacy and government overreach to the surface.”
Finally, the survey shows that IR teams are:
- Detecting the attackers faster than before, with a drastic improvement in dwell time
- Containing incidents more rapidly
- Relying more on in-house detection and remediation mechanisms
To that end, what concepts and methods further enable handlers and investigators as they continue to strive for faster detection and containment? Data science and visualization sure can’t hurt. How can we be more creative to achieve “deeper functionality”? I propose a two-part series on Deeper Functionality for Investigators with R with the following DFIR Redefined scenarios:
- Have you been pwned?
- Visualization for malicious Windows Event Id sequences
- How do your potential attackers feel, or can you identify an attacker via sentiment analysis?
- Fast Frugal Trees (decision trees) for prioritizing criticality
R is “100% focused and built for statistical data analysis and visualization” and “makes it remarkably simple to run extensive statistical analysis on your data and then generate informative and appealing visualizations with just a few lines of code.”
With R you can interface with data via file ingestion, database connection, APIs and benefit from a wide range of packages and strong community investment.
From the Win-Vector Blog, per John Mount “not all R users consider themselves to be expert programmers (many are happy calling themselves analysts). R is often used in collaborative projects where there are varying levels of programming expertise.”
I propose that this represents the vast majority of us, we’re not expert programmers, data scientists, or statisticians. More likely, we’re security analysts re-using code for our own purposes, be it red team or blue team. With a very few lines of R investigators might be more quickly able to reach conclusions.
All the code described in the post can be found on my GitHub.
Have you been pwned?
This scenario I covered in an earlier post, I’ll refer you to Toolsmith Release Advisory: Steph Locke’s HIBPwned R package.
Visualization for malicious Windows Event Id sequences
 |
| Figure 1 |
 |
| Figure 2 |
Taking related log data, parsing and counting it for visualization with R would look something like Figure 3.
 |
| Figure 3 |
 |
| Figure 4 |
How do your potential attackers feel, or can you identify an attacker via sentiment analysis?
- twitteR: provides access to the Twitter API. Most functionality of the API is supported, with a bias towards API calls that are more useful in data analysis as opposed to daily interaction.
- Rtweet: R client for interacting with Twitter’s REST and stream API’s.
- #DDOS
- #Android
- #WireX
 |
| Figure 5 |
The result is a pair of graphs color coded by tweets and retweets per Figure 6.
 |
| Figure 6 |
This gives you an immediate feels for spikes in interest by day as well as time of day, particularly with attention to retweets.
 |
| Figure 7 |
The result in the scenario ironically indicates that the majority of related tweets using our hashtags of interest are coming from Androids per Figure 8. 🙂
 |
| Figure 8 |
orig$text[which.max(orig$emotionalValence)] tells us that the most positive tweet is “A bunch of Internet tech companies had to work together to clean up #WireX #Android #DDoS #botnet.”
orig$text[which.min(orig$emotionalValence)] tells us that “Dangerous #WireX #Android #DDoS #Botnet Killed by #SecurityGiants” is the most negative tweet.
Interesting right? Almost exactly the same message, but very different valence.
How do we measure emotional valence changes over the day? Four lines later…
filter(orig, mday(created) == 29) %>%
ggplot(aes(created, emotionalValence)) +
geom_point() +
geom_smooth(span = .5)
 |
| Figure 9 |
Another line of questioning to consider: which tweets are more often retweeted, positive or negative? As you can imagine with information security focused topics, negativity wins the day.
Three lines of R…
ggplot(orig, aes(x = emotionalValence, y = retweetCount)) +
geom_point(position = ‘jitter’) +
geom_smooth()
…and we learn just how popular negative tweets are in Figure 10.
 |
| Figure 10 |
There are cluster of emotionally neutral retweets, two positive retweets, and a load of negative retweets. This type of analysis can quickly lead to a good feel for the overall sentiment of an attacker collective, particularly one with less opsec and more desire for attention via social media.
In Part 2 of DFIR Redefined: Deeper Functionality for Investigators with R we’ll explore this scenario further via sentiment analysis and Twitter data, as well as Fast Frugal Trees (decision trees) for prioritizing criticality.
Let me know if you have any questions on the first part of this series via @holisticinfosec or russ at holisticinfosec dot org.
Cheers…until next time.
The post toolsmith #128 – DFIR Redefined: Deeper Functionality for Investigators with R – Part 1 appeared first on Security Boulevard.
