Detection Engineering Series: Detections -> Alerts
In the previous post, we talked about Detection Engineering as a discipline within cybersecurity, the vital role and the required skill set to thrive from a high level.
Detection Engineering Series: Detection Engineering Explained
In today's post, we'll discuss the nuances of detections and alerts, key concepts pivotal to the role. After all, you can’t have Detection Engineering without Detections🥁
Detection - A closer look
At the heart of Detection Engineering lies the concept of a "detection."
In simple terms a detection is custom logic, written to spot specific patterns or anomalies that may suggest a security threat. The output is an Alert. This is where the detection ends up being routed to, notifying of a hit.
The full workflow could look like this
Now, this is just one example to give the broader picture but let's dissect the concept of detections and alerts further.
Detection Mechanisms
A detection system comprises a broad range of formats like KQL (Kusto Query Language), SPL (Search Processing Language), Sigma rules, JSON (JavaScript Object Notation), YAML (YAML Ain't Markup Language), YARA (Yet Another Recursive Acronym), or custom Python scripts, to name a few.
These formats, each with their strengths and constraints, are chosen based on the nature of the data, the environment, and the specific threat models a detection engineer is working against. Trust me, there are people who swear by one over the other, but at the end of the day it is better to be agnostic and adaptable to several detection mechanisms.
Understanding and navigating these formats effectively is a fundamental part of a Detection Engineer's role.
Detections could operate at multiple levels within an organization's system, from the user endpoint level to the network level to application level.
For instance, at the user level, detections might be designed to identify abnormal user behavior, such as logging in from unusual locations (remote work would like a word), or downloading content from file sharing sites.
On a network level, the detection system might look for suspicious patterns such as unusual data transfers east to west, abnormally high network connections, or known malicious incoming traffic.
At the application level, the detections could be designed to identify unexpected changes in the system configuration, or signs of malicious code execution. An example could be a production instance spun up without proper change management or usual deployment process (CI/CD, etc.).
The Precision of Detections
Designing a detection requires a precise balance: they need to be wide enough to identify genuine threats but specific enough to not raise an alarm for every slight deviation from the norm (false positives).
This balance is crucial; a system with too many false positives can cause 'alert fatigue,' where genuine threats might be overlooked due to the high number of alerts.
I would argue false positives can be worse than false negatives to a team’s morale but that’s a conversation for another time.
Alerts - The Signals Amidst the Noise
Alerts, in essence, are the product or output of detections. Once the detection logic identifies a potential threat that matches, it triggers an alert. This could be a Slack alert, Teams, email, Jira ticket, and so forth.
As alluded to earlier, the alert system must be configured in a way that can prioritize these signals based on their severity and potential impact. Otherwise, this would be akin to having a car alarm that goes off every time a pedestrian passes by (I’ve been there) – your team would be inundated with ‘false positives’, leading to alert fatigue and potentially missing genuine threats. This is unfortunately quite common in a lot of companies.
What you want in the end state is signal, alerting you when something is wrong. There should be a differentiation between alert and a report of some kind.
Adapting to Evolving Threats
Finally, with cyber threats evolving rapidly, the underlying logic also needs to be adaptive and continuously refined.
Detection Engineers are tasked with staying abreast of the latest threat trends, understanding new vulnerabilities, and modifying detections accordingly to ensure they stay effective. This usually requires continuous tuning and review.
In summary, a detection system is extremely important to an organization's line of defense against cyber threats, and crafting these detections is a task that combines technical expertise, creative problem-solving, and continuous review.
Conclusion
The complexities of detection logic and alert systems require a deliberate approach by Detection teams.
The balance between detecting genuine threats and avoiding false positives, the need for regular tuning, the importance of testing, and the constant requirement for adapting underscore the dynamic nature of this role.
Stay tuned for the next post on this series, and as always, I hope this helped you learn something new.