Playbooks on Velociraptor - Digging deeper!

Finding Files

Mon, 01 Jan 0001 00:00:00 +0000

Scenario

One of the most common operations in DFIR is searching for files efficiently. When searching for a file, we may search by filename, file content, size or other properties.

Main takeaways

This technique can be used to find files on the endpoint.
Turning this into a hunt can search for a file across the entire fleet in minutes.
You can transfer the content of the file or just detect its presence.

Steps to take

The most common artifacts to use are:

Windows.Search.FileFinder or Linux.Search.FileFinder
Windows.NTFS.MFT

More details

The Windows.Search.FileFinder is the Swiss army knife of file searching.

The File Finder artifact

You can use the File Finder artifacts to find files by several criteria:

File names: Adding one or more Glob expressions (i.e. expressions with wildcards) will search for files by name.
In addition to the filename, you can restrict the file by times - by adding a time box (before and after times), only files with any of their timestamps inside the time box will be selected.

This allows us to restrict questions to “only files added in the past week”
You can search file content by adding a Yara rule. Yara is a simple matching language allowing sophisticated search expressions.

By pressing ? inside the Yara editor, Velociraptor will suggest a Yara template you can use to get started.

Matching files can be uploaded to the server. NOTE: This may generate a lot of traffic so it is only suitable for a small number of very targeted matches. Alternatively simply calculate the hash of matching files.

The artifact also supports Windows Volume Shadow Copies, if available. Velociraptor will automatically deduplicate VSS so only a single file match will be reported, even if the same file exists in multiple VSS snapshots - unless the file is changed between them.

Although the artifact is named File Finder, the artifact can also be used to search for registry keys and values. This is because Velociraptor accesses files by way of an accessor. The accessor abstracts access for files and file-like objects.

In Velociraptor the registry accessor makes the registry appear as a filesystem: Registry keys appear as directories and Registry Values appear as Files (with binary content).

This allows the File Finder to search the registry as well - simply change the Accessor option from auto to registry to search the registry. Remember, top level directory is the registry hive, for example HKLM, or HKEY_LOCAL_MACHINE

Example: Detect persistence

Office executables like WINWORD.exe look for AI.exe under the %ProgramFiles%\Microsoft Office\root\<Office Version> and %ProgramFiles(x86)%\Microsoft Office\root\<Office Version> directories. An attacker may place a malicious AI.exe there in order to have persistence whenever a user interacts with the Microsoft Office Suite. 1, 2

Search for file glob C:\Program File*\Microsoft Office\root\Office*\ai.exe and hash it.

Example: Detect registry keys

The .NET DLLs listed in the DOTNET_STARTUP_HOOKS environment variable are loaded into .NET processes at runtime. 1, 2

Solution: Change the Accessor to “registry” and search for the following globs.

HKEY_USERS\*\Environment\DOTNET_STARTUP_HOOKS
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Environment\DOTNET_STARTUP_HOOKS

Select Upload file to view the content of the matching values.

Example: Detect Mark of the Web files

When files are downloaded from the internet, browsers will add an Alternate Data Stream (ADS) to the file called Zone.Identifier. This stream will contain additional information about where the file was downloaded from.

Solution:

Change the accessor to ntfs which will allow us to see Alternate Data Streams (ADS).
Search for files containing the Zone.Identifier (ADS names are appended to the filename with a colon): C:\Users\*\Downloads\**\*:Zone.Identifier
Enable uploading to get the data.
You can preview the data within the GUI in the Uploaded Files tab.

You can also use Windows.NTFS.ADSHunter or Exchange.Windows.Detection.ZoneIdentifier to further parse the contents of the Zone.Identifier stream.

Performance

Searching for files can be an expensive operation, using a lot of CPU and IO resources on the end point. Some rules of thumb:

Searching by filename is fairly cheap.
It is better if you can narrow down the number of directories we have to scan. For example rather than search C:/**/*:Zone.Identifier, limit to search to only user files C:/Users/**/*:Zone.Identifier
It is expensive to search the content of files with a Yara rule.

For example instead of searching for C:/** then applying a Yara rule looking for an executable, it is better to first narrow the glob to C:/**/*.exe and even time box it further. The File Finder artifact prioritizes cheaper operations like filename or modification time checks before attempting to apply a Yara scan.

Preserving Forensic Evidence

Mon, 01 Jan 0001 00:00:00 +0000

Scenario

As a system administrator you have a high level of confidence a certain endpoint is compromised. You wish to preserve critical evidence while arranging for a more experienced DFIR professional to examine the evidence.

Main takeaways

This technique is a “shotgun” approach - it typically collects a lot of data.
Not very targeted.
Does not scale to many endpoints - use when very confident an endpoint is compromised.
DFIR skill required - LOW.

Steps to take

Some common artifacts that are used in a preservation collection:

Windows.KapeFiles.Targets - collects files such as $MFT, log files etc.
Windows.Memory.Acquisition - collects a physical memory image.

If Velociraptor is already installed on the endpoint

For collecting from a single endpoint

Search for the client you want to acquire from the main search screen.
Select Collected Artifacts from the sidebar, then select New Collection from the toolbar.
For Windows systems, select Windows.KapeFiles.Targets.
Select the BasicCollection or Kape Triage or SANS Triage targets. These targets select the most common raw files used by DFIR analysts.
This artifact can collect a large amount of data, we recommend to place resource limits to ensure that collection is not too excessive.

Selecting the Windows.KapeFiles.Targets artifact

If you will have a number of endpoints to collect you can use a hunt to combine the data together:

In the Hunt Manager screen, create a new hunt. Restrict the hunt to a label acquisition.
Proceed to collect the same artifacts as above
As the investigation proceeds, when you find a client that warrants a preservation collection, simply add a label acquisition to the client. The Windows.KapeFiles.Targets will be automatically scheduled on this client.
You can view all acquired clients in the hunt overview page, and create a large zip export of all preserved files.

Make sure this hunt is restricted by labels! If it is not, it will be scheduled on all clients and may result in a lot of data being transferred to the server - this can result in the server’s bandwidth being saturated, disk becoming full or server becoming unresponsive.

If this happens you can stop the hunt in the GUI which will cancel all in-flight collections and eventually recover the server’s bandwidth.

If Velociraptor is not installed on the endpoint

If Velociraptor is not already installed on the endpoint, you can use an offline collector. The Velociraptor Offline Collector is a preconfigured version of Velociraptor which can run on the endpoint and write all raw files into a ZIP archive.

Creating an offline collector

You can use a number of methods to push the offline to collection to the endpoint:

Use an existing EDR or endpoint security product to run the binary and retrieve the collected ZIP file.
Use Group Policy to schedule a task that runs the collector from a remote share.
Use WinRM or PsExec to run the collector remotely (by careful of pass the hash type attacks though).

We recommend using the X509 encryption scheme to store the raw data to an encrypted container. This helps to protect it in transit.

Encrypting the collected data

You can also configure the offline collector to automatically upload the collection to a cloud storage, such as S3, Google Cloud Storage or a Windows file share.

When to use this technique?

This technique should be used sparingly, usually targeting few systems, as it can transfer a large amount of data.

Typically this technique is used when the endpoint is likely to be destroyed (e.g. re-imaged) or become inaccessible in the near future. The method is most suitable for preservation of raw data.

When not to use this technique?

This workflow is not appropriate for triage. A triage workflow is about discovering unknown compromised endpoints and therefore by definition must be applied to a large number of endpoints. The Preserving Forensic Evidence workflow can not be applied to a large number of endpoints.

The traditional digital forensics workflow consists of the following steps:

Acquisition step collects a large amount of raw data to a central server.
Analysis step applies parsers and various dedicated tools in a central “forensics workstation” on all the raw data collected.
Interpretation step involves examining the output from the various parsers to answer case specific questions.

Many newcomers to Velociraptor have been trained with this workflow and try to apply it to Velociraptor.

However this centralist approach to digital forensics does not scale in practice. When trying to apply this approach to a large network of endpoints, users are often overwhelmed with data (it is almost a right of passage for new Velociraptor users to completely fill their server’s disk because they tried to collect all the log files from all the endpoints)

Velociraptor’s philosophy is different:

Velociraptor considers the endpoint to be the ultimate source of truth.
Therefore we rarely need to fetch the raw data from the endpoint! Since Velociraptor can parse the raw data directly on the endpoint we prefer asking case relevant questions from the entire network directly.

To explain this point, let’s consider an example where we want to know if an executable file was recently downloaded onto the endpoint. We will attempt to parse the USN Journal to answer this question.

The very traditional Digital Forensics process (i.e. back in the 90s) would require acquiring a bit for bit disk image of the suspected endpoint with a write blocker attached! Probably no-one does this any more!

A more modern approach is to use a digital acquisition tool to copy just the C:\$Extend\$UsnJrnl:$J file in it’s entirety for Acquisition, then apply a separate tool to parse the journal and search for any files written to the disk with a .exe extension.

While this approach is better than acquiring the entire disk image (Average disks are of the range of 1-2Tb these days), it still requires collecting several hundred MB of data from each endpoint. If you have 10,000 endpoints this quickly becomes intractable.

Velociraptor’s approach is different: Since the endpoint is considered the source of truth, we just directly run our analysis on the endpoint. Collecting the Windows.Forensics.Usn artifact with a FileNameRegex filter of .exe$ will query each machine to parse their own local copy of the USN journal to find all executables directly.

We can collect this artifact using a hunt from the entire network at once. Since each endpoint will be doing the analysis in parallel, results will return in minutes regardless of how many endpoint there are!

This is why we consider simple acquisition followed by central analysis workflows to be inferior to directly collecting the right artifacts.

You should use the Preserving Forensic Evidence workflow only on systems that are known to have been compromised and you want to preserve the raw files for some reason.

Other considerations

When collecting raw files from the endpoint we need to make a tradeoff:

Collecting fewer files may miss some files that are needed later during analysis.
Collecting more files will result in larger collection.

Ultimately it is not possible to know in advance what type of evidence may be relevant. For example, we might collect the $MFT in our initial acquisition but subsequent analysis can show that certain files are critical (e.g. files in the user’s Downloads directory).

In a Preserving Forensic Evidence workflow we can not go back to the source of truth and check the files in the Downloads directory!

Therefore it is always better to have Velociraptor already installed on the endpoint so we can pivot during the analysis phase and get additional information as required.

Triaging Logs

Mon, 01 Jan 0001 00:00:00 +0000

Scenario

An endpoint is suspected of being compromised but you dont know exactly what happened. You want to get an initial idea by examining the logs on the actual endpoint.

Main takeaways

This technique is similar to forwarding logs to a SIEM and applying signatures.
However we can choose very noisy signatures here
We use stacking to quickly categorize the types of activity that happens on the endpoint.

Steps to take

Some common artifacts that are used for Triaging Logs

Windows.Hayabusa.Rules should be imported using Server.Import.CuratedSigma
Exchange.Windows.EventLogs.Hayabusa should be imported from the artifact exchange.

Importing Windows.Hayabusa.Rules

Select the Server Artifacts from the sidebar.
Add a collection, search for Server.Import.CuratedSigma and import the Windows.Hayabusa.Rules. This will import the latest version of the artifact.

This artifact uses the built in Sigma Engine in Velociraptor. The artifact packages the curated Hayabusa rules in a convenient artifact. Rules are categorized by RuleLevel and RuleStatus which generally try to balance how noisy a rule against its detection efficacy.

Sigma workflow in Velociraptor

Because we are trying to triage the endpoint, we actually want to see all the hits, even if they are noisy. We will apply stacking later to quickly triage the type of activity on the endpoint. So in this case we should select to apply all the rules.

Applying all Sigma Rules

Once the artifact is collected from the endpoint we can stack the hits in the GUI:

Update the notebook to remove the LIMIT 50. This will select all rows in one table. Typically there should be many thousands of rows because we added all the noisy rules.
Sort by the rule Title. Hover the mouse on the column header and click the Sort button.
Once the column is sorted, the stacking button should appear.

Stacking a column

Clicking the stacking button will show a summary of the different rules matching and a count of how many times each rule made a hit.

Viewing column summaries

Clicking on any of the rules navigates the table to the list of rules that actually hit.

Viewing common rows

Using this technique it is possible to quickly identify the types or categories of activity on the endpoint and see the most suspicious rules. Due to the stacking we dont need to review each individual hit, but only the different types of rules.

For example, say we see a rule description a PsExec lateral movement, we can quickly identify if PsExec is expected for this environment, or does it represent a potential threat. If I identify the rule as important, I can then review each instance to get more information about what commands were run.

Using the Exchange.Windows.EventLogs.Hayabusa

The Exchange.Windows.EventLogs.Hayabusa artifact is available in the artifact exchange. This artifact uses an external binary Hayabusa to evaluate the Sigma rules from the Hayabusa project.

Post processing and analysing the results from this artifact is similar to the procedure described above.

Discussion and limitations

This technique is similar to many SIEMs which forward event logs from the endpoint and apply matching rules. There are some fundamental differences though:

SIEMs typically only forward a small subset of logs since the more logs are collected the more data the SIEM backend needs to handle. Typically SIEMs forward logs such as Sysmon Process Execution but do not forward other logs for example BITS Client Operational Logs.
SIEM rules are also written to ensure they have a low false positive rate. This means that suspicious activity in one environment which is common in another setting, might not trigger a detection. By stacking on all noisy rules we get to decide for ourselves if a particular rule is acceptable for this environment.

For example an administrator RDP login may be perfectly normal in some environments but a red flag in others! SIEM detections are rarely tuned to the environment.
A SIEM may not be present or well tuned, in a particular environment. Running the Sigma log triaging workflow can compensate for the lack of a SIEM.
Data retention is different from a SIEM. Typically SIEMs only maintain logs for limited time (sometimes as low as a month). On the other hand log files are typically rotated based on size. This means that sometimes logs will be present on the endpoint for longer than in the SIEM, while other times the SIEM will contain logs that were already rotate on the endpoint.
Because this technique relies on locally stored logs, it is susceptible to logs being cleared by attackers.

Real time monitoring

As an additional step you can enable the Windows.Hayabusa.Monitoring artifact for real time detection of the same Sigma rules. This can provide coverage for any future compromises