In this post in our Intro to Incident Response series, we’re going to focus on investigating user logins to find suspicious user activity. This helps to identify user accounts that may have had their credentials compromised, focus the investigation on certain time frames, and identify other computers that are compromised.
We’ll first quickly review the series so far, then dive into user logins, and then see an example using the incident response tool Cyber Triage.
30-Second Refresher
This series is about both a framework for approaching investigations and specific techniques. Let’s review the framework.
We are breaking up the general “Is this computer compromised?” question into three (still big) questions:
- Is there suspicious user activity (such as sensitive files being accessed)?
- Are there malicious programs (such as malware)?
- Are there malicious system changes (such as relaxed security settings)?
And each of those three questions are broken into smaller questions. Eventually, you get to questions that can be answered with data.
At this point early in the series, we are looking at the first bullet: user activity. We break it into three medium-sized questions:
- Who: Which users were active on this system?
- When/Where: Are there suspicious logins to the system? If so, from where and when did they occur?
- What: What did the users do when they were logged into the system?
The last post covered the first bullet, this post covers the second (login activity), and the next will cover the third (user actions).
Why We Investigate User Logins in Incident Reponse
It is common for attackers to obtain passwords (or at least hashes) via phishing or password dumping attacks. With those, they can then log into systems without needing to install malware.
There are several reasons we want to investigate logins:
- Identify accounts that could have been compromised. From there, we can reset passwords and focus on activity for specific accounts (which is the topic of the next post).
- Identify time frames of activity. When we have a suspicious login, we can then use that time frame to look at other network and login activity.
- Identify other hosts. If the suspicious login was a remote login, then we can either investigate the host if it is internal to or block traffic from it if it is external to our network.
Let us consider some scenarios:
- Lateral Movement: When an intruder has gained access to a corporate network, they may brute force passwords or steal hashes so that they can login to other systems in the network. By doing this, they will get closer to the sensitive data they want to steal and to make it harder to trace. In this situation, the intruder could be logging into the computer using an account that is normal for that computer or they maybe using a domain account that would not normally be logging into that computer.
- Common Password: Some managed devices, such as Point of Sale (POS) systems, have a common remote access password so that the 3rd party POS service provider can login to fix the system. That common password can be leveraged by attackers to gain access to all of the systems that are managed by that provider.
- Insider: An authorized employee could be stealing your corporate secrets and accessing data they should not be. They could be logging in at non-standard hours to access the information.
With those scenarios in mind, we’re going to be looking at the history of remote and local logins on a system.
Finding Suspicious User Logins During Incident Response
Following our approach of breaking big questions into smaller questions, we need to break the question, “is there suspicious login activity?,” into smaller and more concrete questions that we can answer with observable data:
- User: Are there users logging in who don’t have a business need to do so?
- This can happen when someone is moving laterally to get closer to an asset.
- To answer this question, you’ll need to know who should be using that computer and compare that with the list of users who have logged in.
- History: Did a user just start logging into this system?
- This can also happen when someone is moving laterally.
- To answer this question, you need to figure out the first login date for each account, which may come from log files or home directory timestamps.
- Remote Host: Is an expected user remotely logging in from an unexpected system?
- This is also a sign of lateral movement and can occur if someone is logging into several internal computers.
- To answer this question, you’ll need to understand how each user typically remotely logs in (if at all) and compare that with the list of remote hosts from login data.
- Time: Is an expected user logging in at an unexpected time?
- This can happen if the account has been taken over and the attacker is logging in when they know the computer owner is not using it.
- To answer this question, you’ll need to know the normal behavior of the user. If you have enough historical log data, you maybe able to learn what the long-term behavior is and look for more recent outliers.
- Brute Force: Are there signs of excessive failed logins?
- This could be a sign that a intruder is trying to guess a password or use the wrong password for a user. (Or, it could be a sign that someone forgot their password).
- To answer this question, you need to look at the logins and track how many failed logins in a row occurred.
- Application: Are users logging in using unexpected remote access applications?
- This can happen if the intruders have installed a remote access application (that is still generating audit logs), or if the intruders are using remote desktop when a different application is normally used.
- To answer this question, you need to review the audit logs and identify the unique applications that are used for logins.
In addition, you should try to answer these questions about outbound connections:
- Destination: Are local users logging in to unexpected computers from this computer?
- This can happen when the account is being used for lateral movement and the current computer is a middle hop point.
- To answer this question, you need to identify the outbound locations and compare those with what you know about the local user.
- Remote User: Did the local user change login names when they logged into the remote system?
- This can happen when the intruder has multiple compromised accounts.
- To answer this question, you need to have access to what remote account they used.
- Times and history are also useful for outbound connections, but we have not found reliable logs recording this.
Like many things with incident response, one of the biggest challenges to answering questions is knowing what is normal for the system and the users. A baseline of normal is needed to know which are abnormal.
Where We Find User Login Data for Incident Response?
Now that we know what questions we want to answer, we need to make sure we have access to the data that can answer them.
To answer all of the above questions, we’ll need to know:
- Who is currently logged into this system.
- Who previously logged into this system (inbound).
- Where did users log into from this system (outbound).
Active Users
To determine who is currently logged into the system, you can use:
- Windows APIs, which can also be accessed by PowerShell / WMI.
- Various tools that use the previously mentioned APIs.
- Analysis of raw memory.
If rootkits have been installed on the system, the Windows APIs may hide the fact that the intruder is actually logged in. Analysis of the raw memory (from a full image or from live analysis) will allow you to remove this risk.
Investigating Inbound User Logins During Incident Response
In Windows, there are three major places to get information about local and remote logins:
- Security Event Log: Contains events from the OS for successful and failed logins (both local and remote). The types and number of events will depend on the OS audit policies. This log may have days or weeks of events. It depends on various audit settings.
- Terminal Services Log: Contains logs from the Terminal Services application that enables remote logins. This typically has events from only successful inbound and local logins (whereas Security also contains failed login attempts), but this log lasts for much longer. I have a system that has remote login events from 8 years ago!
- Memory: A memory dump may have some event logs that were loaded by the OS.
You can get access to these Windows event logs by the local Event Viewer application or by exporting the logs and using an offline tool. But, you will likely have trouble copying the logs from C:WindowsSystem32Config because they will be in use by the OS. We use The Sleuth Kit in Cyber Triage to copy the event logs and therefore bypass the locks imposed by the OS.
Investigating Outbound User Logins During Incident Response
In Windows, there are two ways to get basic outbound login information:
- Security Event Log: Type 4648 events will show when a user used different credentials and needed to authenticate with the remote system.
- Registry: The NTUSER.dat hive has data from the remote desktop client that stores the remote hosts and user names that were previously used. Unfortunately, it is not a log and therefore you can only learn about unique host and user names.
How To Analyze the User Logins
Once you have access to the previously listed data, you’ll have hundreds or thousands of login events. The challenges are:
- Organizing the data so that it is manageable
- Knowing what is normal (such as common remote hosts and finding the unique user names).
Some responders will use Excel; others use search engines; and others review them manually or with their own scripts.
Reviewing User Logins in Using Incident Response Software Cyber Triage
If you are using Cyber Triage, then you can answer these login questions using two parts of the UI. We’ll assume you’ve already collected data from the suspected system using one of our agentless collection methods.
The navigation menus on the left-hand side of Cyber Triage are organized based on the same ideas that this blog series uses. The first items are all about users. The next set is about malware. The last is about system configuration.
For the questions outlined above, we’re going to focus on the “Users” and “Remote Login” sections.
Users Area
We talked a bit about the Users area in the last post when we talked about enumerating the users to get an idea about system usage. We can also answer some of the questions in this post with the data presented. This table shows what accounts exist on the system and what users logged in:
It is populated based on registry data and event logs. From this table, we can find out the user login history, what programs they executed, and more. Refer to the previous post if you are unfamiliar with this area in the UI.
With this data, we can answer:
- Are there users logging in who don’t have a business need to do so? To answer this, review the list of users and compare that with what is known about the use of the system.
- Did a user just start logging into this system? To answer this, refer to the “Earliest Activity” column for each account and compare that with other known activity from the incident.
- Are there signs of excessive failed logins? Cyber Triage will automatically mark accounts as suspicious if they have too many failed sequential logins.
For each account that you think is compromised or want to tag for follow up, select it and mark it as “Bad” or “Suspicious” and add an optional comment:
If you need more context about an account to make a decision, you can always use the bottom area that shows what programs the user ran, processes that were running, etc.:
And the timeline on the right that shows events associated with the incident:
Remote User Logins Area
The second relevant area of Cyber Triage for investigating user logins is “Remote Logins.” This area merges data from various event logs and registry locations into a consolidated table of activity. It groups all remote logins by user and remote host so that it is easier to identify unexpected activity:
With this data, you can answer these questions:
- Is an expected user remotely logging in from an unexpected computer? To answer this, review the groups and expand as needed for further details to see individual logins. The country associated with an external IP will be shown, which can be useful for POS investigations.
- Are local users logging in to unexpected computer from this computer? Outbound logins are shown alongside inbound logins, so you can identify these suspicious outbound hosts when looking for suspicious inbound hosts.
- Did the local user change login names when they logged into the remote system? Cyber Triage will automatically mark logins as suspicious if the outbound login changes account names.
- Are users logging in using unexpected remote access applications? To answer this, look at the “Process” column and look for unexpected programs.
When you are looking at a specific login, you can use the bottom area to get more context about the user and remote host. The Users panel will show you how old the account is and the user’s normal login behaviors (what times they login, how many times per week they login, etc.):
Example User Login Investigation During Incident Response
Let’s look at an example data set. We ingested data from an endpoint and Cyber Triage flagged a user account (“jdoe”) as suspicious because it had outgoing remote desktop (RDP) connections to over 6 different computers:
We know “jdoe” is the primary user of the computer, so the existence of the account is expected. But, the outbound logins are not. We can see them in the bottom area:
They are to HR, sales, and other computers that “jdoe” should not be accessing. We mark the account as “Bad” and assume it has been compromised:
Cyber Triage also marked some logins as suspicious. So, we review those. They are outbound connections that were flagged because the remote user account was different from the local account.
For example, we can see that when the local “jdoe” account remotely logged into dc1 it used the account of domain_admin:
We can mark that activity as bad too since “jdoe” should not have access to the domain_admin account. We should also start an investigation on the dc1 computer based on this activity, which can be done in the background using the Team version of Cyber Triage.
Besides the logins that Cyber Triage marked as suspicious, we should review the other data ourselves because Cyber Triage does not know about expected usage of the system.
We review the hosts associated with inbound connections and find some that are other endpoints in the network and not VPN access points, which is how “jdoe” would typically remotely log in. We mark those as “Bad” and start collections on them.
Conclusion
Analyzing user logins during incident response is an important step to understanding if a user account was compromised and if a system was used for lateral movement. In this post, we broke the question of “is there suspicious login activity” into smaller questions that we can find data to answer.
If you’d like to try Cyber Triage to answer questions about users or malware, download a free evaluation copy here.
