You dont have javascript enabled! Please enable it! Update Microsoft Sentinel Analytics Rules At Scale - CHARBEL NEMNOM - MVP | MCT | CCSP | CISM - Cloud & CyberSecurity

Update Microsoft Sentinel Analytics Rules at Scale

16 Min. Read

Updated—17/04/2024 — The automation tool below was updated to version 1.8. The comparison of rule template versions has been improved to look for objects that aren’t the same because, in some environments, the same rule can be deployed/activated more than once. Hence, the activated rule version will return more than one value. In this case, we don’t create additional rules with the same name and logic if the returned rule version(s) matches at least the latest template version.

Updated—12/04/2024 — The automation tool below was updated to version 1.7. Instead of comparing KQL query changes to update the analytic rule, it compares template versions. So, any changes made to the analytic rule template should also be updated in the active analytic rule. This improves performance and minimizes errors.

Microsoft Sentinel comes with Content Hub, which you can use out-of-the-box to get content value and start on Microsoft Sentinel quickly. Solutions in Microsoft Sentinel Content Hub provide a consolidated way to acquire Microsoft Sentinel content, like data connectors, playbooks, workbooks, analytics rules, and automation in your workspace with a single deployment step.

Updating the Analytics rules in Microsoft Sentinel can be tedious, especially if you have many Content Hub solutions installed. This can be particularly challenging for managed security service providers (MSSPs) who serve multiple tenants from different customers and have many analytic rules to update. Updating these rules every week or month can be time-consuming and boring, right?

In this article, we will show you how to update Microsoft Sentinel Analytics Rules at scale automatically using PowerShell and REST API.

Update Microsoft Sentinel Analytics Rules

Microsoft Sentinel is a cloud-native Security Information Event Management (SIEM) and Security Orchestration Automated Response (SOAR) solution. Microsoft Sentinel delivers intelligent security analytics and threat intelligence across the enterprise, providing a single solution for alert detection, threat visibility, proactive hunting, and threat response.

Content Hub is designed to provide a consistent and scenario-driven approach for onboarding out-of-the-box (OOTB) content as needed. This is accomplished by organizing solutions into packages that comprise data connectors, analytics rules, hunting queries, parsers, playbooks, workbooks, and watchlists. These solutions assist enterprise security operations (SecOps) teams in managing their business, from data ingestion to security monitoring, issue detection, threat hunting, and breach response, all in a scenario-driven mode. With filters to easily discover content for domain categories, content types, support, and more, these solutions enable SecOps teams to handle their business operations more efficiently.

When you update Content Hub solutions, you may notice that Microsoft also updates its built-in analytics rule templates, so if you have created active rules based on those built-in templates, you will see a new version available with the [UPDATE] text added to the Analytic rule name, as shown in the figure below.

UPDATE - Active Analytics Rules
UPDATE – Active Analytics Rules

Then, you can review and compare all the changes for the existing (on the left, in red) and latest (on the right, in green) versions and update these active rules as needed, as shown in the figure below.

Compare the analytics rule to the latest version
Compare the analytics rule to the latest version

Related: Check how to automate Microsoft Sentinel Content Hub Updates.

Microsoft and technology partners regularly update those analytics rule templates. The question that often comes up is: Is there a way to automatically update active analytics rules so I don’t have to set a reminder every week/month to go through and check to see which one has a new updated template version?

As you probably agree, this is a tedious manual operation in the Azure portal. At the time of this writing, there is no out-of-the-box native component that alleviates this manual process.

Related: Check how to enable Microsoft Sentinel Analytics Rules at Scale.

Let’s see how to automate this process and update Microsoft Sentinel active analytics rules at scale.

Credit: I want to give a shout-out to a dear friend and follower, Willem-Jan van Esschoten, for building upon my previous scripts (creating Microsoft Sentinel Analytics Rules at scale and automating Microsoft Sentinel Content Hub updates) to share his work to update all active analytic rules if there’s an update from the Content hub. Thank You!

Prerequisites

To follow this article, you need to have the following:

1) Azure subscription – If you don’t have an Azure subscription, you can create one here for free.

2) Log Analytics workspace – To create a new workspace, follow the instructions to create a Log Analytics workspace.

3) Enable Microsoft Sentinel at no additional cost on an Azure Monitor Log Analytics workspace for the first 31 days; follow the quick onboarding process. Once Microsoft Sentinel is enabled on your Azure Monitor Log Analytics workspace, every GB of data ingested into the workspace can be retained at no charge for the first 90 days.

4) Azure PowerShell is installed locally on your machine or using Cloud Shell.

To install Azure Accounts PowerShell modules on your machine, you can run the following command:

# Install and update to the latest Az PowerShell module
Install-Module -Name Az.Accounts -AllowClobber -Force

# Check the Az PowerShell modules version installed
Get-Module -Name Az.Accounts -ListAvailable | Select Name, Version

5) Before running the script below, you must have active analytics rules and install one or more solutions from Content Hub.

6) The script from the following article (Automate Microsoft Sentinel Content Hub Updates) should be executed first before updating the analytics rules as described below. You could schedule both scripts to run at different times, as follows:

  • 1 – Automate Microsoft Sentinel Content Hub Updates.
  • 2 – Update Microsoft Sentinel Analytics Rules at Scale.

7) Last, ensure you have permission for the Microsoft Sentinel Contributor role, which is required to create and update analytics rules.

Assuming you have all the prerequisites in place, take the following steps:

Update Analytics Rules

This section will describe how to automatically update Microsoft Sentinel Active Analytics Rules for the solution(s) you imported and installed. This tool will work for analytic rules installed from a Content hub solution and standalone analytic rules.

You have several options for running the script: Azure Cloud Shell, Visual Studio Code, or Windows Terminal. The Script works with PowerShell 5.1 or PowerShell 7 (core) with the Az module.

.EXAMPLE-1

.\Update-AnalyticsRules.ps1 -SubscriptionId "SUB-ID" -ResourceGroup "RG-Name" `
     -WorkspaceName "Log-Analytics-Name" -skipTunedRulesTextInput "DNS" -Verbose

This example will connect to your Azure account using the subscription ID, resource group name, log analytics name, and, optionally, the KQL query code you specified; then, it checks for active and enabled analytics rules and filters the ones that require an update. The “-skipTunedRulesTextInput” parameter takes a keyword in the KQL query to prevent the rule from being updated.

Here is an example of the output once you run this tool:

Update Microsoft Sentinel Analytics Rules
Update Microsoft Sentinel Analytics Rules

Track Updates

Apr 9, 2024 (version 1.5): The automation tool can be improved if Microsoft updates the name, description, severity, techniques, sub-techniques, etc., to the rule template(s) without updating the KQL query. However, we initially opted for the changes made in the KQL query by Microsoft or third-party providers, which are more important than the name, description, severity, techniques, etc. So, in version 1.5, if Microsoft or third-party providers update the detection KQL query, we automatically update the active Analytics Rules; the ones you tuned or modified their KQL logic are untouched.

Apr 12, 2024 (version 1.7): The automation tool was updated to not depend on comparing KQL query changes to update the analytic rule; it compares template versions. So, any changes made to the analytic rule template should also be updated in the active analytic rule, including entities, techniques, sub-techniques, etc. This improves performance and minimizes errors. The Analytics rule will be updated once Microsoft or third-party providers release a new analytic rule template version.

Apr 17, 2024 (version 1.8): The automation tool below was updated to improve the comparison of rule template versions to look for objects that aren’t the same. In some environments, the same rule can be deployed/activated more than once, so the activated rule version will return more than one value. In this case, we don’t create additional rules with the same name and logic if the returned rule version(s) matches at least the latest template version.

This is version 1.8. If you have any feedback or changes that everyone should receive, please feel free to leave a comment below.

//Area of improvement//

Suppose you previously set the (OOB) analytic rule severity to be low, and the new update (by Microsoft or Third-Party Providers) sets the latest (OOB) rule to high severity. In that case, the script will generally default to the latest version, which is high severity for the analytic rule, and update it.

Suppose you want to keep your current tuning to the (OOB) rule (like severity, techniques, sub-techniques, etc.) when a new version is updated too. In that case, you need to add additional comparison(s)/condition(s) similar to the current one for template version comparison (see below) to update the analytic rule KQL query and keep your current tuning/changes untouched.

As a workaround, if you made any changes in the rule, like changing the severity, entity, sub-techniques, or something else, you can add a comment line in the Kusto query with the specific “keyword” and an explanation of what has changed. Then, after running the script, the parameter “-skipTunedRulesTextInput” prevents the rollback of the changes.

PowerShell Code

Below is the script to automatically update built-in Microsoft Sentinel active Analytics Rules installed Content Hub solutions. You need to run this script on demand; check the next section to learn how to automate the update process based on a weekly schedule.

<#
.SYNOPSIS
Update all default Microsoft Sentinel active Analytics Rules at once. Tuned or modified analytic rules are untouched.

.DESCRIPTION
How to update built-in Microsoft Sentinel active Analytics Rules at once using PowerShell and REST API.

.NOTES
File Name : Update-AnalyticsRules.ps1
Author    : Microsoft MVP/MCT - Charbel Nemnom and Willem-Jan van Esschoten
Version   : 1.8
Date      : 04-April-2024
Updated   : 18-April-2024
Requires  : PowerShell 6.2 or PowerShell 7.x.x (Core)
Module    : Az Module

.LINK
To provide feedback or for further assistance please visit:
https://charbelnemnom.com

.EXAMPLE
.\Update-AnalyticsRules.ps1 -SubscriptionId <SUB-ID> -ResourceGroup <RG-Name> `
    -WorkspaceName <Log-Analytics-Name> -skipTunedRulesTextInput <Skip-Tuned-Analytics-Rules> -Verbose
This example will connect to your Azure account using the subscription Id specified, and then update all active analytics rules from templates.
By default, only the rules with the state Enabled will be updated.
#>

param (
    [Parameter(Position = 0, Mandatory = $true, HelpMessage = 'Enter Azure Subscription ID')]
    [string]$subscriptionId,
    [Parameter(Position = 1, Mandatory = $true, HelpMessage = 'Enter Resource Group Name where Microsoft Sentinel is deployed')]
    [string]$resourceGroupName,
    [Parameter(Position = 2, Mandatory = $true, HelpMessage = 'Enter Log Analytics Workspace Name')]
    [string]$workspaceName,
    [Parameter(Position = 3, Mandatory = $false, HelpMessage = 'Enter a Keyword that exists in the KQL query to prevent the rule from updating')]
    [string]$skipTunedRulesTextInput
)

# Add Wildcards to Skip Tuned/Modified Analytics Rules Text Input
$skipTunedRulesText = "*$($skipTunedRulesTextInput)*"

#! Install Az Module If Needed
function Install-Module-If-Needed {
    param([string]$ModuleName)

    if (Get-Module -ListAvailable -Name $ModuleName) {
        Write-Host "Module '$($ModuleName)' already exists, continue..." -ForegroundColor Green
    }
    else {
        Write-Host "Module '$($ModuleName)' does not exist, installing..." -ForegroundColor Yellow
        Install-Module $ModuleName -Force -AllowClobber -ErrorAction Stop
        Write-Host "Module '$($ModuleName)' installed." -ForegroundColor Green
    }
}

function Update-AnalyticRule {
    param (
        $rulePayload,
        $apiVersion,
        $ruleName,
        $ruleDisplayName
    )
    # Define analytic rule URI
    $ruleURI = "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.OperationalInsights/workspaces/$workspaceName/providers/Microsoft.SecurityInsights/alertRules/$($ruleName)$($apiVersion)"
    try {
        $ruleResult = Invoke-AzRestMethod -Method PUT -path $ruleURI -Payload $rulePayload -Verbose:$false

        if (!($ruleResult.StatusCode -in 200, 201)) {
            Write-Host $ruleResult.StatusCode
            Write-Host $ruleResult.Content
            throw "Error when updating Analytic rule: $($ruleDisplayName)"
        }        
    }
    catch {
        Write-Error $_ -ErrorAction Continue
    }
    return $ruleResult
}

function Update-Metadata {
    param (
        $metadataPayload,
        $apiVersion,
        $ruleName,
        $ruleDisplayName
    )
    # Define metadata URI for the Analytic rule
    $metadataURI = "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.OperationalInsights/workspaces/$workspaceName/providers/Microsoft.SecurityInsights/metadata/analyticsrule-$($ruleName)$($apiVersion)"
    try {
        $resultMetadata = Invoke-AzRestMethod -Method PUT -path $metadataURI -Payload $metadataPayload -Verbose:$false

        if (!($resultMetadata.StatusCode -in 200, 201)) {
            Write-Host $resultMetadata.StatusCode
            Write-Host $resultMetadata.Content
            throw "Error when updating Metadata for Analytic rule: $($ruleDisplayName)"
        }
        else {
            Write-Verbose "Updating Metadata for Analytic rule: $($ruleDisplayName)"
        }
    }
    catch {
        Write-Error $_ -ErrorAction Continue
    }
}

#! Install Az Accounts Module If Needed
Install-Module-If-Needed Az.Accounts

#! Check Azure Connection
Try {
    Write-Verbose "Connecting to Azure Cloud..."
    Connect-AzAccount -ErrorAction Stop | Out-Null
}
Catch {
    Write-Warning "Cannot connect to Azure Cloud. Please check your credentials. Exiting!"
    Break
}

# Define the latest API Version to use for Sentinel
$apiVersion = "?api-version=2024-03-01"

#! Get Az Access Token
$token = Get-AzAccessToken #This will default to Azure Resource Manager endpoint
$authHeader = @{
    'Content-Type'  = 'application/json'
    'Authorization' = 'Bearer ' + $token.Token
}

# Get all installed content Rule Templates
Write-Verbose "Get all installed content Rule Templates..."
$contentURI = "https://management.azure.com/subscriptions/$subscriptionid/resourceGroups/$resourceGroupName/providers/Microsoft.OperationalInsights/workspaces/$workspaceName/providers/Microsoft.SecurityInsights/contentTemplates$($apiVersion)"
$contentResponse = (Invoke-RestMethod $contentURI -Method 'GET' -Headers $authHeader).value

try {
    $contentTemplates = $contentResponse | Where-Object { $_.properties.contentKind -eq "AnalyticsRule" }
    if ($contentTemplates.Count -eq 0) {
        throw "No content Rule templates can be found. Please check and install Analytics Rule from the Content Hub blade"
    }
}
catch {
    Write-Error $_ -ErrorAction Stop
}

# Get all active Analytics Rules
Write-Verbose "Get all active Analytics Rules..."
$activeRulesURI = "https://management.azure.com/subscriptions/$subscriptionid/resourceGroups/$resourceGroupName/providers/Microsoft.OperationalInsights/workspaces/$workspaceName/providers/Microsoft.SecurityInsights/alertrules$($apiVersion)"
$ruleResponse = (Invoke-RestMethod $activeRulesURI -Method 'GET' -Headers $authHeader).value
# Filter only enabled Analytics rules
Write-Verbose "Filter only enabled Analytics rules..."
$ruleEnabled = $ruleResponse.properties | Where-Object enabled -eq True

# Filter out tuned and modified KQL rules (Optional)
if ($skipTunedRulesText -ne "**") {
    Write-Verbose "Filter out tuned and modified KQL rules..."
    $ruleSkipTuned = $ruleEnabled | Where-Object { $_.query -notlike $skipTunedRulesText }
    $activeRules = $ruleSkipTuned | Where-Object alertRuleTemplateName -ne $null | Select-Object alertRuleTemplateName
}
else {
    $ruleSkipTuned = $ruleEnabled | Where-Object { $_.query -like $skipTunedRulesText }
    $activeRules = $ruleEnabled | Where-Object alertRuleTemplateName -ne $null | Select-Object alertRuleTemplateName
}

# Filter out Content Rule Templates to match active, enabled, and skipped tuned/modified Analytics rules
Write-Verbose "Filter out Content Rule Templates to match active and enabled Analytics rules..."
$matchedTemplates = $contentTemplates | Where-Object {
    $template = $_
    $activeRules.alertRuleTemplateName -eq $template.properties.contentId }

Write-Output "$($matchedTemplates.count) matched Active Analytics Rules were found!"

$updatedActiveRules = @()

foreach ($contentTemplate in $matchedTemplates) {

    # Get the latest Template of the active Analytics Rule    
    $ruleTemplateURI = "https://management.azure.com/subscriptions/$subscriptionid/resourceGroups/$resourceGroupName/providers/Microsoft.OperationalInsights/workspaces/$workspaceName/providers/Microsoft.SecurityInsights/contentTemplates/$($contentTemplate.name)$($apiVersion)"
    $ruleResponse = Invoke-RestMethod $ruleTemplateURI -Method 'GET' -Headers $authHeader -Verbose:$false
    $ruleContentId = $ruleResponse.properties.contentId
    $ruleProperties = $ruleResponse.properties.mainTemplate.resources | Where-Object type -eq 'Microsoft.OperationalInsights/workspaces/providers/metadata' | Select-Object properties
    $ruleProperties.properties = $ruleProperties.properties | Select-Object * -ExcludeProperty description, parentId
    Write-Verbose "Getting the latest Template of the Active Analytic Rule => $($ruleResponse.properties.mainTemplate.resources.properties.displayName)"   

    # Comparing the latest rule template version to active Analytics Rule version
    $templateVersion = $ruleResponse.properties | Select-Object version
    $ruleTemplateVersion = $ruleSkipTuned | Where-Object alertRuleTemplateName -eq $ruleContentId | Select-Object templateversion    
    Write-Verbose "Comparing the latest rule template version $($templateVersion.version) to Active Analytic Rule version $($ruleTemplateVersion.templateVersion)"    

    # If the version is not in (the objects aren't the same), update the analytic rule
    if ($templateVersion.version -notin $ruleTemplateVersion.templateVersion) {
        Write-Verbose "Analytic rule requires an update, updating the analytic rule..."

        # 'Microsoft.SecurityInsights/AlertRuleTemplates' for analytic rules installed from a Content hub solution
        # 'Microsoft.OperationalInsights/workspaces/providers/alertRules' for standalone analytic rules
        $rule = $ruleResponse.properties.mainTemplate.resources | Where-Object { $_.type -eq 'Microsoft.SecurityInsights/AlertRuleTemplates' -or $_.type -eq 'Microsoft.OperationalInsights/workspaces/providers/alertRules' }
               
        # Load the latest Analytic rule properties to JSON
        if (!$rule.properties.alertRuleTemplateName) {
            $rule.properties | Add-Member -NotePropertyName alertRuleTemplateName -NotePropertyValue $rule.name
        }
        if (!$rule.properties.templateVersion) {
            $rule.properties | Add-Member -NotePropertyName templateVersion -NotePropertyValue $ruleResponse.properties.version
        }
        $rule.properties.enabled = $true 
        $rulePayload = $rule | ConvertTo-Json -Depth 100

        # Update Active Analytic Rule
        Write-Verbose "Updating Analytic rule: $($rule.properties.displayName)" 
        $ruleResult = Update-AnalyticRule -rulePayload $rulePayload -apiVersion $apiVersion -ruleName $rule.name -ruleDisplayName $rule.properties.displayName

        # Load the Metadata properties to JSON
        $ruleResult = $ruleResult.Content | ConvertFrom-Json -Depth 100
        if (!$ruleProperties.properties.parentId) {
            $ruleProperties.properties | Add-Member -NotePropertyName parentId -NotePropertyValue $ruleResult.id
        }
        $metadataPayload = $ruleProperties | ConvertTo-Json -Depth 100
        # Update Metadata for Active Analytic Rule
        Update-Metadata -metadataPayload $metadataPayload -apiVersion $apiVersion -ruleName $rule.name -ruleDisplayName $rule.properties.displayName

        $updatedActiveRules += $rule
    }
}

if ($updatedActiveRules.count -eq 0) {
    Write-Output "All the active Analytics Rules are currently up to date. No update is required."
}
else {
    Write-Output "$($updatedActiveRules.count) Active Analytics Rules were found and updated!"
}

Automate Microsoft Sentinel Analytics Rules Updates

The next step is automating the updates to the Analytics Rules using Azure Automation Accounts, Azure Logic Apps, or Azure Functions. In this example, we will look at automating this update process using Automation Accounts.

You might ask why Automation Accounts instead of Azure Logic Apps or Azure Functions? Creating Runbooks as part of process automation is easier to configure, and Automation Accounts is cheaper than Logic App or Function App.

First, we must create an Azure automation resource with a Managed Identity (or use an existing one). Microsoft recommends using Managed Identities for the Automation accounts instead of Run As accounts. Managed identity would be more secure and offer ease of use since it doesn’t require any credentials to be stored. Azure Automation support for Managed Identities is now generally available.

//Azure Automation Run as accounts (including Classic Run as accounts) retired on 30 September 2023 and was replaced with Managed identities.

Create Automation Account

Creating an Automation Account creates a new service principal in Microsoft Entra ID (formerly Azure AD) by default. Next, you must assign the appropriate (Azure RBAC) role to allow access to Microsoft Sentinel for the service principal at the resource group level where Sentinel is deployed.

In this example, we have assigned the Microsoft Sentinel Contributor role to the managed identity at the resource group level. When assigning permissions, always remember to use the principle of least privilege (PoLP).

Add role assignment to Automation Account Managed Identity
Add role assignment to Automation Account Managed Identity

If you have an existing Automation Account with system-assigned managed identity enabled, skip this step and jump to the “Create PowerShell Runbook” section.

Using a system-assigned managed identity for an Azure Automation account
Using a system-assigned managed identity for an Azure Automation account

Open the Azure portal and click All Services in the upper left-hand corner. In the list of resources, type Automation. As you begin typing, the list filters based on your input. Select Automation Accounts.

Click +Add. Enter the automation account name, and choose the right subscription, resource group, and location. A system-assigned managed identity is selected by default, as shown in the figure below.

Create an Automation Account with Managed Identity
Create an Automation Account with Managed Identity

Then select Review + Create and click Create.

Create PowerShell Runbook

In this step, you create a PowerShell Runbook to update Microsoft Sentinel Analytics Rules automatically. You can directly edit the runbook’s code using the text editor in the Azure portal. Alternatively, you can use any offline text editor, such as Visual Studio Code, and import the Runbook into Azure Automation.

From your automation account, select Runbooks under Process Automation. Click the ‘+ Create a runbook‘ button to open the Create a runbook blade, as shown in the figure below.

Please make sure to select the right runtime PowerShell version. This example will use the PowerShell Az modules targeting the 7.2 runtime version.

Create PowerShell runbook runtime version 7.2
Create PowerShell runbook runtime version 7.2

Next, select Review + Create and click Create.

Edit the Runbook

Once you have created the runbook, you must edit it and add the script below. As mentioned earlier, we will create a Runbook to connect to your Azure account using the subscription ID, resource group name, log analytics name, and, optionally, the KQL query code you specified; then, it checks for active and enabled analytics rules, filters the ones that require an update, and apply the update.

The automation runbook is as follows. It is a modified version of the above script fine-tuned for Automation Accounts:

<#
.SYNOPSIS
Update all default Microsoft Sentinel active Analytics Rules at once. Tuned or modified analytic rules are untouched.

.DESCRIPTION
How to update built-in Microsoft Sentinel active Analytics Rules at once using PowerShell and REST API.

.NOTES
File Name : Update-AnalyticsRules.ps1
Author    : Microsoft MVP/MCT - Charbel Nemnom and Willem-Jan van Esschoten
Version   : 1.8
Date      : 04-April-2024
Updated   : 18-April-2024
Requires  : PowerShell 6.2 or PowerShell 7.x.x (Core)
Module    : Az Module
Service   : Automation Accounts

.LINK
To provide feedback or for further assistance please visit:
https://charbelnemnom.com

.EXAMPLE
.\Update-AnalyticsRules.ps1 -SubscriptionId <SUB-ID> -ResourceGroup <RG-Name> `
    -WorkspaceName <Log-Analytics-Name> -skipTunedRulesTextInput <Skip-Tuned-Analytics-Rules> -Verbose
This example will connect to your Azure account using the subscription Id specified, and then update all active analytics rules from templates.
By default, only  the rules with the state Enabled will be updated.
#>

param (
    [Parameter(Position = 0, Mandatory = $true, HelpMessage = 'Enter Azure Subscription ID')]
    [string]$subscriptionId,
    [Parameter(Position = 1, Mandatory = $true, HelpMessage = 'Enter Resource Group Name where Microsoft Sentinel is deployed')]
    [string]$resourceGroupName,
    [Parameter(Position = 2, Mandatory = $true, HelpMessage = 'Enter Log Analytics Workspace Name')]
    [string]$workspaceName,
    [Parameter(Position = 3, Mandatory = $false, HelpMessage = 'Enter a keyword that exists in the KQL query to prevent the rule from being updated')]
    [string]$skipTunedRulesTextInput
)

# Ensures you do not inherit an AzContext in your runbook 
Disable-AzContextAutosave -Scope Process 

#! Check Azure Connection
Try {
    Write-Output "Connecting to Azure Cloud..."
    # Connect to Azure with system-assigned managed identity (automation account)
    Connect-AzAccount -Identity -ErrorAction Stop | Out-Null
}
Catch {
    Write-Warning "Cannot connect to Azure Cloud. Please check your managed identity Azure RBAC access. Exiting!"
    Break
}

# Set Azure Subscription context
Set-AzContext -Subscription $subscriptionId

# Define the latest API Version to use for Sentinel
$apiVersion = "?api-version=2024-03-01"

# Add Wildcards to Skip Tuned/Modified Analytics Rules Text Input
$skipTunedRulesText = "*$($skipTunedRulesTextInput)*"

function Update-AnalyticRule {
    param ( 
        $rulePayload,
        $apiVersion,
        $ruleName,
        $ruleDisplayName        
    )    
    # Define analytic rule URI
    $ruleURI = "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.OperationalInsights/workspaces/$workspaceName/providers/Microsoft.SecurityInsights/alertRules/$($ruleName)$($apiVersion)"
    try {
        $ruleResult = Invoke-AzRestMethod -Method PUT -path $ruleURI -Payload $rulePayload -Verbose:$false

        if (!($ruleResult.StatusCode -in 200, 201)) {
            Write-Host $ruleResult.StatusCode
            Write-Host $ruleResult.Content
            throw "Error when updating Analytic rule: $($ruleDisplayName)"
        }        
    }
    catch {
        Write-Error $_ -ErrorAction Continue
    }
    return $ruleResult    
}

function Update-Metadata {
    param ( 
        $metadataPayload,
        $apiVersion,
        $ruleName,
        $ruleDisplayName        
    )
    # Define metadata URI for the Analytic rule
    $metadataURI = "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.OperationalInsights/workspaces/$workspaceName/providers/Microsoft.SecurityInsights/metadata/analyticsrule-$($ruleName)$($apiVersion)"
    try {
        $resultMetadata = Invoke-AzRestMethod -Method PUT -path $metadataURI -Payload $metadataPayload -Verbose:$false

        if (!($resultMetadata.StatusCode -in 200, 201)) {
            Write-Host $resultMetadata.StatusCode
            Write-Host $resultMetadata.Content
            throw "Error when updating Metadata for Analytic rule: $($ruleDisplayName)"
        }
        else {
            Write-Output "Updating Metadata for Analytic rule: $($ruleDisplayName)"
        }        
    }
    catch {
        Write-Error $_ -ErrorAction Continue
    }    
}

#! Get Az Access Token
$token = Get-AzAccessToken #This will default to Azure Resource Manager endpoint
$authHeader = @{
    'Content-Type'  = 'application/json'
    'Authorization' = 'Bearer ' + $token.Token
}

# Get all installed content Rule Templates
Write-Output "Get all installed content Rule Templates..."
$contentURI = "https://management.azure.com/subscriptions/$subscriptionid/resourceGroups/$resourceGroupName/providers/Microsoft.OperationalInsights/workspaces/$workspaceName/providers/Microsoft.SecurityInsights/contentTemplates$($apiVersion)"
$contentResponse = (Invoke-RestMethod $contentURI -Method 'GET' -Headers $authHeader).value

try {    
    $contentTemplates = $contentResponse | Where-Object { $_.properties.contentKind -eq "AnalyticsRule" }
    if ($contentTemplates.Count -eq 0) {
        throw "No content Rule templates can be found. Please check and install Analytics Rule from the Content Hub blade"
    }
}
catch {
    Write-Error $_ -ErrorAction Stop
}

# Get all active Analytics Rules
Write-Output "Get all active Analytics Rules..."
$activeRulesURI = "https://management.azure.com/subscriptions/$subscriptionid/resourceGroups/$resourceGroupName/providers/Microsoft.OperationalInsights/workspaces/$workspaceName/providers/Microsoft.SecurityInsights/alertrules$($apiVersion)"
$ruleResponse = (Invoke-RestMethod $activeRulesURI -Method 'GET' -Headers $authHeader).value
# Filter only enabled Analytics rules
Write-Output "Filter only enabled Analytics rules..."
$ruleEnabled = $ruleResponse.properties | Where-Object enabled -eq True
 
# Filter out tuned and modified KQL rules (Optional)
if ($skipTunedRulesText -ne "**") {
    Write-Output "Filter out tuned and modified KQL rules..."
    $ruleSkipTuned = $ruleEnabled | Where-Object { $_.query -notlike $skipTunedRulesText }
    $activeRules = $ruleSkipTuned | Where-Object alertRuleTemplateName -ne $null | Select-Object alertRuleTemplateName
}
else {
    $ruleSkipTuned = $ruleEnabled | Where-Object { $_.query -like $skipTunedRulesText }
    $activeRules = $ruleEnabled | Where-Object alertRuleTemplateName -ne $null | Select-Object alertRuleTemplateName
}

# Filter out Content Rule Templates to match active, enabled, and skipped tuned/modified Analytics rules
Write-Output "Filter out Content Rule Templates to match active and enabled Analytics rules..."
$matchedTemplates = $contentTemplates | Where-Object { 
    $template = $_
    $activeRules.alertRuleTemplateName -eq $template.properties.contentId }

Write-Output "$($matchedTemplates.count) matched Active Analytics Rules were found!"

$updatedActiveRules = @()

foreach ($contentTemplate in $matchedTemplates) { 
   
    # Get the latest Template of the active Analytic Rule    
    $ruleTemplateURI = "https://management.azure.com/subscriptions/$subscriptionid/resourceGroups/$resourceGroupName/providers/Microsoft.OperationalInsights/workspaces/$workspaceName/providers/Microsoft.SecurityInsights/contentTemplates/$($contentTemplate.name)$($apiVersion)"
    $ruleResponse = Invoke-RestMethod $ruleTemplateURI -Method 'GET' -Headers $authHeader -Verbose:$false
    $ruleContentId = $ruleResponse.properties.contentId
    $ruleProperties = $ruleResponse.properties.mainTemplate.resources | Where-Object type -eq 'Microsoft.OperationalInsights/workspaces/providers/metadata' | Select-Object properties
    $ruleProperties.properties = $ruleProperties.properties | Select-Object * -ExcludeProperty description, parentId
    Write-Output "Getting the latest Template of the Active Analytic Rule => $($ruleResponse.properties.mainTemplate.resources.properties.displayName)"
           
    # Comparing the latest rule template version to active Analytics Rule version
    $templateVersion = $ruleResponse.properties | Select-Object version
    $ruleTemplateVersion = $ruleSkipTuned | Where-Object alertRuleTemplateName -eq $ruleContentId | Select-Object templateversion    
    Write-Output "Comparing the latest rule template version $($templateVersion.version) to Active Analytic Rule version $($ruleTemplateVersion.templateVersion)"

    # If the version is not in (the objects aren't the same), update the analytic rule
    if ($templateVersion.version -notin $ruleTemplateVersion.templateVersion) {
        Write-Output "KQL query requires an update, updating the analytic rule..."       
        
        # 'Microsoft.SecurityInsights/AlertRuleTemplates' for analytic rules installed from a Content hub solution
        # 'Microsoft.OperationalInsights/workspaces/providers/alertRules' for standalone analytic rules
        $rule = $ruleResponse.properties.mainTemplate.resources | Where-Object { $_.type -eq 'Microsoft.SecurityInsights/AlertRuleTemplates' -or $_.type -eq 'Microsoft.OperationalInsights/workspaces/providers/alertRules' }
       
        # Load the latest Analytic rule properties to JSON
        if (!$rule.properties.alertRuleTemplateName) {
            $rule.properties | Add-Member -NotePropertyName alertRuleTemplateName -NotePropertyValue $rule.name
        }
        if (!$rule.properties.templateVersion) {
            $rule.properties | Add-Member -NotePropertyName templateVersion -NotePropertyValue $ruleResponse.properties.version
        }
        $rule.properties.enabled = $true 
        $rulePayload = $rule | ConvertTo-Json -Depth 100

        # Update Active Analytic Rule
        Write-Output "Updating Analytic rule: $($rule.properties.displayName)"   
        $ruleResult = Update-AnalyticRule -rulePayload $rulePayload -apiVersion $apiVersion -ruleName $rule.name -ruleDisplayName $rule.properties.displayName
        
        # Load the Metadata properties to JSON
        $ruleResult = $ruleResult.Content | ConvertFrom-Json -Depth 100
        if (!$ruleProperties.properties.parentId) {
            $ruleProperties.properties | Add-Member -NotePropertyName parentId -NotePropertyValue $ruleResult.id
        }
        $metadataPayload = $ruleProperties | ConvertTo-Json -Depth 100
        # Update Metadata for Active Analytic Rule
        Update-Metadata -metadataPayload $metadataPayload -apiVersion $apiVersion -ruleName $rule.name -ruleDisplayName $rule.properties.displayName       

        $updatedActiveRules += $rule
    }      
}

if ($updatedActiveRules.count -eq 0) {
    Write-Output "All the active Analytics Rules are currently up to date. No update is required."
}
else {
    Write-Output "$($updatedActiveRules.count) Active Analytics Rules were found and updated!"
}

Save the script in the CMDLETS pane, as shown in the figure below.

Edit PowerShell Runbook in Azure Automation
Edit PowerShell Runbook in Azure Automation

Then, test the runbook using the “Test pane” and fill in all the required parameters to verify it works as intended before publishing it.

You must manually supply the following 3 or 4 parameters on the Test page and then click the Start button to test the automation script.

  1. SUBSCRIPTIONID: Mandatory
  2. RESOURCEGROUPNAME: Mandatory
  3. WORKSPACENAME: Mandatory
  4. SKIPTUNEDRULESTEXTINPUT: Optional
Test Automate Microsoft Sentinel Analytics Rules Updates
Test Automate Microsoft Sentinel Analytics Rules Updates

Once the test is completed successfully, you must publish the Runbook by clicking the Publish button. This is a very important step.

Schedule the Runbook

In the final step, you can schedule the runbook to run when you want to check for updates to Analytics Rules.

Select Schedules within the same Runbook you created in the previous step and click “+ Add Schedule.”

So, if you need to schedule the Runbook to run every week, create the following schedule with Recur every 1 Week, pick the desired day of the week, Set the expiration to No, and then click “Create.” At the minimum, you can run the Runbook every 1 hour, and you can also run it on-demand if you wish to do so.

Add a schedule for Runbook
Add a schedule for Runbook

While scheduling the Runbook, you must enter the parameters for the PowerShell script to run successfully. In this scenario, you need to specify the following 3 parameters:

  • Azure Subscription ID: Where the Sentinel instance is provisioned.
  • Resource Group: Where Microsoft Sentinel is deployed.
  • Workspace Name: The Log Analytics Workspace name.
  • Skip Tuned Rules Text Input: Enter a keyword in the KQL query to prevent the rule from being updated.

The automation script takes those parameters as input, as shown in the figure below.

Add required parameters
Add required parameters

Once done, click OK twice.

Last, you can test and monitor the Runbook from the “Jobs” page of Runbooks under the Resources section. Then, look for the Output, Errors, Warnings, and All Logs, as shown in the figure below.

Runbook Jobs
Runbook Jobs

That’s it, there you have it. Happy Automating and Updating Microsoft Sentinel Analytics Rules at Scale!

In Summary

This article showed you how to use PowerShell and REST API to update Microsoft Sentinel Analytics Rules at scale. This is very useful if you have many analytics rules activated based on the default rule templates, which are installed as part of Content Hub solution(s), and you want to automate the update of the query logic regularly by using Automation Accounts, Azure Logic Apps, or Azure Functions.

Microsoft’s security experts and analysts have created analytics rule templates based on known threats, common attack vectors, and suspicious activity escalation chains. These templates get regularly and automatically updated to search for any suspicious activity across your environment. You can customize the templates to search for activities relevant to your needs. When these rules detect suspicious activity, they generate alerts that create incidents. You can assign and investigate these incidents in your environment.

The power of Microsoft Sentinel comes from the ability to detect, investigate, respond to, and remediate threats.

__
Thank you for reading my blog.

If you have any questions or feedback, please leave a comment.

-Charbel Nemnom-

Photo of author
About the Author
Charbel Nemnom
Charbel Nemnom is a Senior Cloud Architect with 21+ years of IT experience. As a Swiss Certified Information Security Manager (ISM), CCSP, CISM, Microsoft MVP, and MCT, he excels in optimizing mission-critical enterprise systems. His extensive practical knowledge spans complex system design, network architecture, business continuity, and cloud security, establishing him as an authoritative and trustworthy expert in the field. Charbel frequently writes about Cloud, Cybersecurity, and IT Certifications.
Previous

Augment Microsoft Sentinel Incident Investigation with Microsoft Copilot for Security and Logic Apps

Quick Guide to AWS Lambda Pricing

Next

2 thoughts on “Update Microsoft Sentinel Analytics Rules at Scale”

Leave a comment...

  1. Hello Vipul, yes, this is totally feasible to export the JSON of multiple (active) analytics rules!
    After line 188 of the script in this section (PowerShell Code), you need to add another 2 lines of code to export the Active Analytics rules locally on your machine, as follows:

    $ruleExport = $ruleSkipTuned | Where-Object alertRuleTemplateName -eq $ruleContentId | ConvertTo-Json -Depth 100
    $ruleExport | Out-File -FilePath "C:\temp\$($ruleQuery.displayName).json" -Force

    Hope it helps!

Let us know what you think, or ask a question...