Building Custom KQL Analytics Rules in Sentinel

In today’s post, we explore using Microsoft Sentinel to detect, and respond to threats using custom analytics rules using Kusto Query Language (KQL).  This allows you to tailor threat detection to your organization’s specific needs, ensuring that no threat goes unnoticed and tuning the detections to exclude low value and noisy results that clutter up the incident queue.

We will dive into the details of building custom KQL analytics rules in Microsoft Sentinel.  Exploring the data so that we become more comfortable with queries, tuning those queries, and improve our understanding of the data in our own environment.  We will explore several in-depth examples that highlight the flexibility and power of KQL, enabling you to create highly customized and effective threat detection rules.

Understanding KQL Basics

Before we dive into custom rule creation, let’s briefly touch on the basics of KQL.  Kusto Query Language is a powerful query language used to interact with Azure Data Explorer.  It is designed for querying large datasets and can be used to extract, manipulate, and analyze data.

A simple KQL query might look like this:

SecurityEvent
| where TimeGenerated > ago(7d)
| where EventID == 4625
//Exclude accounts that are not relevant to tune this query
| summarize FailedLogins = count() by Account, bin(TimeGenerated, 1h)
| sort by FailedLogins desc

This query retrieves failed login events from the last 7 days, counts them by account, and sorts the results by the number of failed logins.

Building Custom Analytics Rules

Example 1 – Detecting Brute Force Attacks

Brute force attacks are a common threat where attackers attempt to gain access by systematically trying different passwords. Here’s a custom rule to detect such attacks:

SigninLogs
| where ResultType == "50126" //Failed login attempts
| summarize FailedAttempts = count() by UserPrincipalName, bin(TimeGenerated, 1h)
| where FailedAttempts > 10 //Adjust this number as needed
//Tune this query by filtering on UPN as needed to exclude low relevance results
| extend AccountCustomEntity = UserPrincipalName

This rule identifies user accounts with more than 10 failed login attempts within an hour, flagging potential brute force attacks.

Example 2 – Monitoring Suspicious IP Addresses

Monitoring access from suspicious IP addresses is crucial for identifying potential threats. This rule integrates threat intelligence to detect such activity:

let ti_indicators = ThreatIntelIndicator
| where IndicatorType == "IP"
| project IndicatorValue;

SecurityEvent
| where IPAddress in (ti_indicators)
| summarize Count = count() by IPAddress, Account, bin(TimeGenerated, 1h)
| where Count > 5  //Tune the count # and Accounts to fit your own logs
| extend AccountCustomEntity = Account, IPCustomEntity = IPAddress

This query correlates IP addresses from the threat intelligence feed with security events, identifying IPs with frequent activity.

Example 3 – Detecting Data Exfiltration

Data exfiltration is a serious threat where sensitive data is transferred out of the organization. Here’s a rule to detect unusual data transfer activity:

CommonSecurityLog
| where DeviceVendor == "Cisco" and DeviceProduct == "ASA"
| where Message has "data transfer"
| extend DataTransferred = todouble(extract("([0-9]+) bytes", 1, Message))
| summarize TotalDataTransferred = sum(DataTransferred) by SourceIP, bin(TimeGenerated, 1h)
| where TotalDataTransferred > 1000000000 // 1 GB
| extend IPCustomEntity = SourceIP

This rule monitors Cisco ASA logs for data transfer activities exceeding 1 GB within an hour, indicating potential data exfiltration.  Tune this for the correct and logical data transfer size for your own environment.  Different edge protection devices have different field names and messages, so be sure to explore what is needed in your own logs.

Best Practices for Custom Analytics Rules

• Clearly define what you aim to detect with each rule. This helps in crafting precise and effective queries.
• Leverage contextual data, such as user profiles and threat intelligence, to enrich your queries and improve detection accuracy.
• Cyber threats evolve rapidly, so regularly review and update your analytics rules to ensure they remain effective.
• Before deploying a rule, test your KQL queries to verify their accuracy and performance.

Summary

Creating custom KQL analytics rules in Microsoft Sentinel provides the flexibility, customization, and tuning capabilities that organizations require to address evolving threats; and to keep pace with changing infrastructure.

A key piece of analytics in Sentinel is to continue to tune, update, and adapt your queries on a regular basis.  Good Sentinel governance practices do impact the ability of a SOC team, and updating your Sentinel configurations on a regular basis will make a positive impact.

In this article, we’ve explored several examples of KQL queries, showcasing the versatility of Microsoft Sentinel.  By following best practices and regularly updating your rules, you can ensure that your organization is well-equipped to detect and respond to the ever-evolving landscape of cybersecurity threats.  Embrace the power of custom analytics in Microsoft Sentinel and transform your security operations with actionable insights and proactive threat mitigation by continuing to build out detections, automations, and good team communications practices.

To learn more about KQL and using queries in your day-to-day operations, check out https://github.com/rod-trent/MustLearnKQL to take your knowledge to the next level!