Detection Engineering or Threat Hunting?
In a recent post I wrote about Detection Engineering, and what it entails for a practitioner.
At times, Detection Engineering and Threat Hunting can be used interchangeably.
Although there are a lot of similarities, this post will outline some of the differences between the two disciplines. We’ll go over the key differences, strengths, and drawbacks of each.
Detection Engineering
Unlike threat hunting, detection engineering has actually been around for a long time, for example with Intrusion Detection Systems (IDS). Analysts, engineers, and researchers would allocate a lot of resources to sift through logs, to then come up with detection signatures based off of the network traffic from those logs.
However, Detection engineering concepts have matured with time.
What started as detections for atomic indicators, (think IP addresses, domains), modern detection engineering incorporates indicators based on threat actors’ behaviors and tools (TTPs).
Detections undergoes multiple phases before they finally end in signals of malicious activity, aka rules or alerts. These efforts concentrate on known indicators or behavior and serve a specific purpose, creating signals that will then capture malicious activity.
The detection engineering process also differs from threat hunting. Let’s discuss some of these that comprise that process.
One of the most important and time-consuming tasks in detection engineering is dealing with false positives. An overload of false positives can become a reality (ask any person in InfoSec 😂), so tradeoffs have to be made here.
Another important undertaking is analyzing the current state of detections and tuning existing rules based on their performance. This could mean removing a rule if it is necessary. (Even if you were the one who wrote it )
In this way, Detection engineering has a more organizational and methodical process to it.
Similar to the development process, rules undertake a testing and development period before they make it into production. Whether this goes in CI/CD pipeline is ultimately up to each team, but generally that is where things are headed.
Threat Hunting
While Detection Engineering focuses on identifying and responding to known threats or behaviors, Threat Hunting is a proactive process aimed at uncovering hidden, unknown threats that have bypassed your existing security measures. This includes your custom detections.
Threat Hunters actively look for anomalies that could signal a security threat independent of alerts that are configured. This requires a deep understanding of the organization's network, systems, and normal user behavior, allowing them to spot deviations that might indicate a security breach.
Aside from finding something that discovers an incident, Threat Hunting could result in discovering policy violations. This is also worthy of the time investment. Either way, it’s a win- win.
Threat hunting takes a different approach when it comes to querying and analyzing the data. Since threat hunting aims to identify threats that might have evaded detections, an understanding of how current detections are structured is required (knowing what normal looks like).
Depending on the initial hypothesis, the focus can be spent on techniques that are not covered by current detections or techniques too difficult to detect due to noise (high false positive rates).
An example can be searching and analyzing the usage of the least commonly used applications during the past month.
This was just one example, but here is a good article from CyborgSecurity that goes over many other examples in a simplistic manner.
Threat hunting queries can have a wider tolerance for noise in the results.
For more on Threat Hunting, see this post that goes over how to approach it in your environment.
Detection Engineering and Threat Hunting: Two Sides of the Same Coin
While Detection Engineering and Threat Hunting are distinct disciplines, they share common ground.
Both fields require a deep understanding of the threat landscape, strong analytical skills, and proficiency in various security technologies and tools.
There could be different domains of focus, for example NDR (Network Detection & Response), EDR (Endpoint Detection & Response), SIEM (Security Information and Event Management) SOAR (Security Orchestration, Automation and Response)), detection content, and so forth.
In the end, they require a deep understanding of a domain and how to leverage it to prove/disprove a bad thing has happened. (Bit of an oversimplification, but you get the point)
They are intertwined and need aspects of each other to be successful.
Both disciplines aim to identify and mitigate security threats, albeit in different ways.
In a nutshell, here are some of the key differences between the two.
Detection Engineering is typically a more planned and repeatable process, while Threat Hunting is usually more ad hoc and creative.
Detection Engineering is typically focused on identifying known threats or behaviors, while Threat Hunting is typically focused on identifying unknown threats.
Detection Engineering relies heavily on automation, using custom logic to search for and identify threats. Threat Hunting, by contrast, is a (mainly) manual process, requiring human experience to spot anomalies.
Here are some of the strengths and drawbacks of each.
Detection Engineering
Methodical and repeatable
Automated in nature.
Will identify threat designed to detect each time
Will miss unknown threats
Threat Hunting
Ad hoc
Mostly manual in nature
Can identify threats not known to existing detection mechanisms
Can be time-consuming and resource-intensive
Here’s a graphic on how they can fit together.
As you can infer from the image, eventually, Threat Hunts could end up resulting in Detections themselves and vice versa. The data leads to analytics, which leads to hunting, which leads to investigations, and this then leads to detections.
A flow chart from the post The Dotted lines between Threat Hunting and Detection Engineering giving another take on this.
Conclusion
In the end, both Detection Engineering and Threat Hunting are very important aspects to a company's Cybersecurity program. It’s not Detection Engineering vs Threat Hunting, but how you can implement the best of both practices.
By understanding the nuances of each, we can better appreciate their unique contributions and how they work together to create a comprehensive defense against threats.
I hope this helps understand these 2 domains within the umbrella that is Cybersecurity.