This Popular MITRE ATT&CK Metric Is Leading Us Astray
This post was last updated on April 19th, 2021 at 03:26 pm
Why the pursuit for “FULL” coverage isn’t necessary, and what metric you should be concentrating on instead
MITRE ATT&CK is the de facto standard for assessing modern behavioral detection against adversary tactics and techniques. Its power resides not just in providing a common language for attacker behaviors, but also as a historical anthology of what the security community has observed during attacks.
As with any framework, from Lockheed’s Cyber Kill Chain to David Bianco’s Pyramid of Pain, their power is sometimes matched by the misunderstanding and, often, misuse by the security industry. With ATT&CK, the biggest blunder we continue to see is pressing our security teams with the unnecessary and unachievable drive for FULL MITRE COVERAGE™*. I’m here to tell you that that goal is not only unnecessary, it’s probably negatively impacting your detection program and your budget.
*Not an actual trademark
The Infocyte team is not immune to the initial desire for “full” coverage either, our most recent product update (currently in EARLY ACCESS) brought MITRE ATT&CK technique and subtechnique mapping into our Behavioral Analytics Engine. Along the way, we had to make decisions about what techniques we would concentrate on and which to ignore. The initial wisdom always seems to start out as: “more is better”… lets peel that back and see why that’s not true for most of us pursuing practical application.
How We Value The Behaviors to Monitor For
It’s now uncontroversial to say that detecting modern cyber attacks requires modern behavior monitoring (especially on the endpoint/server level). When looking at ATT&CK, there are about 20 or so common behaviors that we see in almost every attack. On the other end, we have many obscure, advanced techniques briefly described in a DEFCON talk or a recent forensics report from a state attack. What’s on page two of those forensics reports and rarely discussed in the DEFCON talk is the abundant usage of the common behaviors the advanced actor ALSO used shortly thereafter or along the way.
The question is: is having more coverage of those rare, obscure techniques we rarely see going to net you the win when the same attack also trips multiple common techniques as well?
Threat Hunters often call these “Bluebird” analytics. We, as an industry, spend a lot of time discussing the latest obscure technique believing that finding that is what’s going to find us the really advanced attacks that we aren’t seeing today.
As most experianced detection pros and threat hunters have learned, they often come up with nothing. I argue that each “Bluebird” analytic written against a rarely seen techniques really just dilutes the time & resources that could be going toward better describing the more common behaviors that are seen in almost every attack, even the advanced ones.
Why do we chase full coverage? Often, when an organization adopts MITRE ATT&CK as a framework, managers will predictably ask for a metric to grade against. The metric many in the industry have collectively choosen is “coverage” with the following definition:
Coverage = Number of techniques we can detect/mitigate divided by the total number of techniques described by MITRE ATT&CK
I argue that this metric will inevitably result in inefficient and wasteful allocation of resources.
Why FULL MITRE ATT&CK Coverage Isn’t Necessary
There are a couple main problems to this coverage metric approach:
- ATT&CK techniques vary greatly in how often they are used.
- Some techniques are rarely used by adversaries.
- Some may have only been historically used once before the threat actors realized it was a poor technique.
- On the other hand, there are common techniques that are used in almost every breach of a Windows environment (e.g. PowerShell execution or credential dumps).
- Almost all attacks, advanced and commodity, exhibit common behaviors
- Even “ZERO DAY” attacks that use a novel entry method, like SolarWinds SUNBURST (Solarigate) or Hafnium’s Exchange vulnerabilities, still end up using common techniques along their attack chain.
- In fact, many zero days are found by observing common attacker behaviors and asking how they got in. (See a suspicious powershell command spawn from MS Exchange’s w3wp.exe process? That’s not normal… time to investigate).
- Good detection requires good signal to noise ratios. ATT&CK provides no information or guidance on expected noise or quality.
- Some techniques are not possible to distinguish between normal network activity. For example, you may want to monitor for ATT&CK T1548.002 – UAC Bypass. There problem is: legitimate admins do this all the time (high noise) and the handful of really evil techniques you could build detection on are overly specific and change often. It can be a futile waste of time and resources to build robust detection coverage on a technique like this.
- Another example is environment specific: Attempting to implement alerting on suspicious PowerShell usage in an Azure DevOps environment where PowerShell is used all the time. Personally, this is not my idea of fun.
- Some techniques are harder to monitor than others.
- Infocyte and other behavior-based EDRs monitor for new process creation events (Execution Tactic) on Windows. On Windows this is easy (< 1000 new processes a day), on a database server or any Linux box this becomes harder (10k+ new processes per day).
- Monitoring new processes is also a lot more performant and achievable than hooking a lot of lower level API calls (T1106), which can happen millions of times per minute. Attempting too much of this is often a reason why your antivirus solution significantly slows your systems down.
- DFIR pros will tell you the hardest thing to determine in an investigation is what was exfiltrated. Collection (TA0009) and Exfiltration (TA0010) tactics have minimal visibility options in most environments–it’s just plain hard to implement practically for most environments.
If you need more evidence that full coverage is unnecessary for effectiveness, just take a look at the tests used to judge modern behavioral detection solutions by MITRE themselves. They don’t use a script that runs through every technique, they use a few real-world adversaries, like APT29, that have demonstrated use of a cross section of key techniques common to other attackers:
Behavioral Monitoring for the Rest of Us
By now it’s become common knowledge that endpoint behavioral monitoring is necessary to detect modern attacks. But unless you have a 24/7 SOC with a full time threat intelligence team, you’re really better off not trying to chase coverage of every attacker behavior possible. Instead, concentrate defenses around the most commonly observed ATT&CK behaviors that are achievable to monitor. These are the ones that actually matter and the ones that will catch more malicious actors, more often.
Report after report on the latest attacks continues to confirm that monitoring all possible behaviors is not necessary to detect advanced attacks. Defense in Depth still works: every tactic and technique you have visibility on is a detection opportunity in the attack chain. We are all strapped for resources; I encourage you to stop chasing the highest coverage and focus on covering the most common ones more robustly.
“Budget concious security teams should ensure they have visibility around the top 20 most commonly observed ATT&CK behaviors”– Chris Gerritz – VP, Threat Intelligence & Response
In a future post in this series, I will show you how if you have visibility on the Top 20 Most Common (Endpoint) Hacker Behaviors, there are exceedingly few attacks that could get past your notice. Additionally, I’ll dive into the details of how Infocyte defends against these techniques using our new Behavioral Analytics Engine and our unique historical forensic capabilities.