5 Tips for Effective Threat Hunting

As each day passes organizations are finding unique challenges as their cyber landscape rapidly evolves.  Data breaches and ransomware activity are on the rise, and some organizations have tried to address the threat landscape with an “automate everything” approach.  While that's a great concept, automation is just one piece of the security puzzle and not the entire picture. We also need humans in the loop who can work alongside automation, or the passive detection mechanisms to dig deeper into the data and find the unpredictable.

This is where threat hunting comes in. A once unique job title in the already specialized world of cybersecurity, threat hunting is now a much-needed skillset for every well-rounded security analyst.  That means organizations must understand what this important function is: roughly defined as the discovery of malicious artifacts, activity or detection methods that are not accounted for in passive monitoring capabilities. In other words, proactively searching for and identifying threats or indicators in the environment that may have gone undetected.

Defining the Hunt
The first things that come to mind are artifacts and activity, for example, evidence of malicious actions and in some cases indications of a current compromise. This is the objective of a hunter, but we should not define a successful hunt as one that always find evidence of an attacker. Instead a successful hunt may also be one that produces a new detection method or indicator that was previously unavailable. To illustrate how threat hunters produce these outcomes let’s look at  five tried and tested tips for threat hunting.

Tip 1. Hypothesize and Test
The proverb goes “The journey of a thousand miles begins with a single step,” and in threat hunting that single step is a question that leads to a hypothesis. Threat hunters must take a step back and ask themselves, “Is this secure?” To answer this question, a threat hunter needs to further ask, “If I were to attack this environment, how would I do it? What assets would I attempt to gain access to? Who would be my targets?” The answers to these questions are the basis of your threat hunting hypotheses.

For example, if a hunter believes a particular machine may be targeted, such as an engineer’s machine hosting valuable code, he or she may begin formulating their theories around how an attacker might attempt to exfiltrate data from this device. They may begin looking for odd services, unusual network connections, abnormal behaviors, or anything that seems out of place for this device or environment. As you uncover more data in testing your original hypothesis, more questions will result that can be used to form a more targeted hypothesis.

In many cases the hunt may not produce positive results but that does not mean the base theory was incorrect. These theories should not be seen as rigid. Instead they are fluid and can be further expanded upon or refined with evidence from the collected data. Perhaps no unusual behaviors were found when only the engineering machines were in scope for the hunt.  When the same queries where then expanded to apply to a larger set of devices, abnormal data was then found.  On the opposite end of the spectrum, if this expansion still does not yield results, then the queries written by the hunter could be automated and tied to an alerting mechanism to produce passive detections. In both situations the hunt has produced valuable results.

Tip 2. Use a Map
You generally do not want to jump in and begin testing these hypotheses blindly.  You may be able to identify malicious activity just by looking through data and seeing what's unusual, but it helps even more to have a framework for mapping discoveries to and categorizing the activity you are seeing. This is where threat-based frameworks like the MITRE ATT&CK framework or the Lockheed Martin Cyber Kill Chain help to fully form your theories and guide you in your testing.

Say you have identified some unusual activity. Where do you go from there? One step forward is to reference the framework to pinpoint exactly where in the kill chain the activity you have identified resides, and what the attacker’s next steps may be. For example, you discovered an unauthorized script in the startup folder of a machine. After examining the contents of the script, you found mechanisms that classify it as “Persistence.” With this information you can then review a framework and see that generally the next step for an attacker having gained “Persistence” will be “Privilege Escalation.” This leads to the formulation of a new hypothesis on how an attacker may escalate their privileges within your environment and gives you a starting point on what to look for.

Tip 3. Maintain a Global Perspective
While frameworks provide a useful guideline, they are relatively static sources of information. To enhance their potential, consider leveraging external intelligence - information outside of your environment that you can use to start a hunt. This can include ‘after action reports’ of another environment, newly published attack methods, or recent threat research. Threat hunters need to stay on top of this news and continually ask themselves how it applies to their environment, or if existing methods could be easily altered to apply to their environment.

Consider the variables in any threat report as an opportunity to form new hypotheses to test. If a new report is issued on a vulnerability in a software suite that could allow attackers to execute arbitrary code, ask yourself how this could apply to your environment. Do you have this vulnerability internally? Could the same vulnerability be applied to a similar software suite? Could the vulnerability be used to execute other actions?

Tip 4. Manage Situational Awareness
There are also many organization-specific concerns that may not be thoroughly addressed by external sources. This is where internal intelligence comes into play.  How well do you know your environment?  Review all the data you have collected in past hunts or any reports that have been generated by your inside incident response team. All of these reports contain valuable information and allow you to ask, “Is there is a chance that something was missed?” Learn from the past and consider how each documented compromise may indicate weaknesses within your environment.

It’s vital to know what your environment looks like:

These are critical questions because you cannot defend something you do not know or can not see. Attackers are going to scan the entire environment and look for weak points, so threat hunters must have holistic visibility of their environment to ensure they are not at a visibility disadvantage. Threat hunters must also familiarize themselves with users and user activity. Understand normal operating hours, what websites are generally visited, or applications colleagues use on a regular basis. All of this helps to filter the data and identify anomalous activity.

Tip 5. Pivot from a Macro to Micro Focus
The last step is to narrow down the focus of the hunt. While the previous steps give threat hunters a view of the larger picture, they still need to drill down from there. This means looking at the network itself to see what's taking place and diving deeper into what's occurring on those endpoints.

To illustrate, in the same way a police officer may observe interstate traffic and then pull over a motorist and inspect their car, threat hunters can look for unusual traffic by testing their hypothesis at the macro level. They may be looking for traffic going to an odd location or an usual number of network connections, or at an odd time, or a network connection with mismatched protocols. These are examples of indicators that will always lead the threat hunter to the actual infected or compromised endpoint. It allows you to observe the network as a single entity and then zoom in on anything suspicious for further investigation.

For instance, maybe you see an account that's been utilized on 13 machines when it should only be on one. Once you find those 13 compromised machines you have a smaller group of endpoints to monitor and investigate, rather than sifting through mountains of data in an attempt to find a threat. Start with a 10,000 foot view, then focus as necessary.

At the end of the day, threat hunting is about proactively testing hypotheses, discovering evidence of threats, and developing the next generation of passive detection methods. As ransomware incidents and advanced persistent threats continue to expose the stress points of traditional detection capabilities, organizations must prioritize proactive, hypothesis-driven discovery in the form of threat hunting as a way of minimizing attacker impact and further securing an environment.

About the Author:
Christopher Fielder, Senior Cybersecurity Strategist, Fidelis Cyber Security

Christopher Fielder is a 20-year veteran of the cyber landscape. His background includes military, government, academic, and corporate defense strategies including active response and threat hunting initiatives.