In the 2nd post in our Intro to IR Triage series, we’re going to take a big picture view. I want to give you the roadmap of how we are going to approach this series before diving into the technical details.
(If you’d like to check out the rest of the series, here’s the series navigation page).
In the last post, we talked about the goal of triage and considerations to make when looking at tools. Now we are going to look at how we answer the triage questions about if a computer was compromised and if you care. The future posts will dive into the specific areas mentioned in this post.
What Are We Looking For?
Host triage wants to answer the investigative questions of “Is this computer compromised?” and “If so, how badly and have I seen it before?”. It’s not trying to get to root cause analysis.
So, how do you answer those questions? Well, it depends.
Evidence could be anywhere on the system. The bad guy (or girl) is trying to blend in so that he isn’t detected. To do this, he’ll use different and evolving techniques to gain and maintain access. He will also probably try to clean up his tracks and not leave evidence behind.
Example evidence may include malware starting from one of dozens of registry keys, malware that took over a normal process to make it do bad things, or there could be no malware and instead a password was stolen. The exact evidence is going to depend on who attacked you, what he was looking for, and your environment.
So, we’re basically looking everywhere…. But, that doesn’t help. So, let’s force some structure on this problem.
Types of Indicators
We all learn that to solve a big problem, then you should break it into smaller problems. This is why we call the approach “Divide and Conquer“. So, I break the problem of answering “Is this computer compromised” into three smaller questions (that we’ll break down more later):
- Are there malicious system changes (such as relaxed security settings)?
- Are there malicious programs (such as malware)?
- Is there suspicious user activity (such as sensitive files being accessed)?
Now, these categories have a bit of overlap because there could be remote user activity from malware or malware can make system changes, but I still think it is a useful breakdown to make it more manageable. If you have an alternative breakdown, let me know.
I’ll also note that these three categories will rely on the same data sources. For example, the Windows registry is going to be used to look for system configuration, malware persistence, and what programs a user ran. So, with these categories we aren’t going to think about registry (or file system) forensics. Instead, we are going to think about how we answer specific questions, such as “were malicious programs executed”.
As we’ll see as this series progresses, one of the challenges to answering these questions is knowing what is normal. We’re looking for suspicious and malicious things. To know that, you need to know what is good (or normal) on those systems. This is often a challenge for incident responders.
Let’s look at each of these medium-sized questions in a bit more detail so that we can talk about smaller-sized questions.
System Configuration Changes
Your default build may not be what the bad guy needs to get his work done. He may not like the fact that you have security software on the system and may not like that he can’t remotely log into the host.
Instead of installing malicious programs on the host, it is often better for the bad guy to use what is there (which may mean reconfiguring what is there). Some refer to this as “living off of the land”.
So, as part of the host triage, you should look for changes. We can break the question of “are there malicious system changes” into smaller questions of:
- Were settings changed to reduce detection or attribution? Such as:
- Disabled firewalls or antivirus programs
- Decreased audit levels
- Were settings changed to preserve their foothold? Such as:
- Creation of local accounts with administrator privileges
- Enabling of remote access
- Enabling of file sharing to copy data to or from this system
- Patching of vulnerable software so that someone else can’t get in
- Disabling OS updates so that they can continue to exploit it
- Where settings changed to prevent recovery? Such as:
- Disabling and deleting of of shadow volume copy so that you can’t restore during a ransomware attack.
To investigate these changes, we’re going to need the registry and other configuration files. You can either manually poke around the control panel on the system or run some tools that collect the data. We’ll discuss that later in the series.
Malicious Programs
Malicious programs (or malware, viruses, APT, etc.) play a role in a lot of incidents because they allow the attacker to exploit the system, maintain access, and extract data from it. So, you should always look for it as part of host triage.
In this category, we are going to be looking for evidence of their current, future, or past execution. We’re going break the question of “Are there malicious programs on the system” into smaller questions of:
- Are there malicious startup / autoruns programs?
- Unless it is a one-time shot, malware is going to want to run on a periodic basis and therefore will have some trigger. We’ll need to look at the places that can start malware.
- Are there malicious running processes?
- Because so many programs look at startup locations, malware can be fairly tricky and it is sometimes easiest to see it when it is running. So, we’ll want to look at running processes to see if a legit process has been taken over by a malicious one. We’ll look at memory, files, network connections, etc.
- Are there remnants of malicious processes?
- It could be that the malware isn’t running when you are there and you don’t see how it could have started. But, you may be able to find evidence that it ran in the past. So, we’ll look for files, DNS cache, and execution history.
Looking for malicious program indicators means you need to look in a lot of places and includes a lot of volatile data. We’ll cover what we are looking for and how to find it later in the series.
User Activity
Lastly, we need to look at user activity. An employee or an attacker with compromised credentials can do a lot of damage without malware or changing the system configuration. They may already have all of the access that they need.
We are going to break the question of “Is there suspicious user activity?” into smaller questions of:
- Is there suspicious login activity? Such as:
- Users logging in at abnormal times
- Users logging in from abnormal locations
- Is the user doing suspicious things? Such as:
- Users running abnormal programs
- Users accessing folders they do not need to
- Users creating encrypted ZIP files
One of the major challenges with looking at user activity is that a full record of user actions is not likely to be found. You may find registry keys that refer to programs that were run, but you may not have time stamps. You may have enabled logging to record when programs were launched, but that is not turned on by default.
Workflow
We’ve broken the bigger problem into three medium-sized problems and further broken them into smaller problems. But, then the question is where to start.
I’d recommend that you start with the medium-sized problem that you suspect the most. If you saw some suspicious network traffic to a Command and Control (C&C) server coming from the system, then I’d start with looking for malicious programs because those are what usually contact C&C servers. If you saw suspicious network traffic based on remote desktop, then start with system configuration and user activity.
A complete triage investigation should cover all of these categories, but you might as well start with the category that could be easiest to find the initial thread of evidence to start pulling on.
Conclusion
Host triage is a big problem that is best approached as a set of smaller problems. I look at it as three medium-sized problems with further smaller problems that can be tackled one-by-one. If you think I missed a smaller question, then let me know.
The next set of posts in this series will look at identifying suspicious user activity. Then we’ll do malware and system configuration. As we’ll see, one of the common challenges with these small problems is knowing what is normal.
The series is going to use Cyber Triage as an example tool (but the techniques and ideas are tool agnostic). If you want to try the eval version of Cyber Triage before then, fill out the form.
