Extract Custom Details from Microsoft Sentinel into Logic App

When a security alert is triggered, the information provided in the alert is vital for the security analyst to conduct an investigation. Therefore, the alert must contain essential details. Including custom information in the alert will enhance the efficiency of the investigation. Custom details enable users to add specific information to the alert generated by scheduled and near-real-time (NRT) alert rules.

This article will show you how to extract custom details from Microsoft Sentinel alerts and incidents into the Logic App to enhance your alerts.

Introduction

Microsoft Sentinel is a cloud-native Security Information Event Management (SIEM) and Security Orchestration Automated Response (SOAR) solution. Microsoft Sentinel delivers intelligent security analytics and threat intelligence across the enterprise, providing a single solution for alert detection, threat visibility, proactive hunting, and threat response.

Scheduled query analytics rules analyze events from data sources connected to Microsoft Sentinel and generate alerts when the contents of these events are noteworthy from a security perspective. These alerts are further analyzed, organized, and filtered by Microsoft Sentinel’s various engines and condensed into incidents that require a SOC analyst’s attention. However, when the analyst views the incident, only the properties of the component alerts themselves are immediately visible. Accessing the actual content – the details contained in the events – requires some investigation.

Using the custom details feature in the analytics rule wizard, you can include event data in the alerts created from those events. This allows the event data to become part of the alert properties, giving you quick access to event content in your incidents. This helps you prioritize, investigate, make conclusions, and respond more efficiently and rapidly.

In the expanded Custom details section, as shown in the figure below, you can add key-value pairs corresponding to the details you want to surface. In the Key field (left-side), you enter a name that will appear as the field name in alerts. In the Value field (right-side), you can choose the event parameter you wish to surface in the alerts from the drop-down list. This list will be populated by values corresponding to the fields in the tables that are the subject of the rule KQL query.

If you are trying to automate response for Microsoft Sentinel incidents that have Custom Entities, you need to pass those Entities to Logic Apps so that the Entity details can be sent to the End user via email using the Logic App Connector; you’ll see that the default “Entities” does not capture the Custom Details that you have defined in the alert rule logic.

In this guide, we’ll illustrate all the steps to extract incident Entities and Custom Details you configured as part of the Analytics Rules using the Microsoft Sentinel incident playbook.

Prerequisites

To follow this guide, you need to have the following:

1) Azure subscription – If you don’t have an Azure subscription, you can create one here for free.

2) Azure and Email Communication Services resources are provisioned in your Azure subscription. Azure Communication Services offers multichannel communication APIs for adding voice, video, chat, text messaging/SMS, email, and more to all your applications. Learn how to create Communication Service and Email Communication Service resources.

3) Log Analytics workspace – To create a new workspace, follow the instructions to create a Log Analytics workspace.

4) Enable Microsoft Sentinel at no additional cost on an Azure Monitor Log Analytics workspace for the first 31 days; follow the quick onboarding process. Once Microsoft Sentinel is enabled on your Azure Monitor Log Analytics workspace, every GB of data ingested into the workspace can be retained at no charge for the first 90 days.

5) One or more Analytics Rules (Scheduled or NRT rule) include Entity Mapping and Custom Details. You can define up to 20 custom details in a single analytics rule. Each custom detail can contain up to 50 values. The combined size limit for all custom details and their values in a single alert is 2 KB. Values over this limit are dropped.

Related: Check how to enable Microsoft Sentinel Analytics Rules at Scale.

6) Last, ensure you have the following roles assigned on the following resources:

• Microsoft Sentinel Contributor on the Log Analytics workspace where Microsoft Sentinel is deployed.
• Logic App Contributor on whichever resource group will contain the playbook created in this guide.

Assuming you have all the prerequisites in place, take the following steps:

Create a playbook with an incident trigger

First, we need to create a playbook with an incident trigger:

1) For Microsoft Sentinel in the Azure portal, select the Configuration > Automation page. For Microsoft Sentinel in the Defender portal, select Microsoft Sentinel > Configuration > Automation.

2) On the Automation page, select Create > Playbook with incident trigger.

3) In the Create playbook wizard, under Basics, select the subscription and resource group, and give the playbook a name.

4) Select Next: Connections >. Under Connections, the Microsoft Sentinel – Connect with managed identity connection should be visible, as shown in the figure below.

5) Select Next: Review and create >.

6) Under Review and create, select Create and continue to designer. The Logic app designer opens a logic app with the name of the playbook, as shown in the figure below.

Select Related Entities

In the Logic app designer, under the step where you want to select related entities, select New step and then click Add an action.

Under Add an action, in the search box, type ‘Select‘ as your filter. From the actions list, choose Select under Data Operations.

Provide this information about your selection:

From

• For the value, select Insert Expression ‘fx‘, and then under Dynamic content, start typing ‘entities‘ and select Entities.

Map

• For the first value, type Entity and then enter ‘item()?['properties']?['friendlyName']‘ under Function.
• For the second value, type Entity Type and then enter ‘item()?['kind']‘ under Function.

Create HTML table with Related Entities

Once we have selected the related entities, we will send the output to an HTML table.

In the Logic app designer, under the step where you added the select related entities, select New step and then click Add an action.

Under Add an action, in the search box, type ‘HTML‘ as your filter. From the actions list, choose Create HTML table under Data Operations.

Provide this information about your selection:

From

• For the value, select Insert Expression ‘fx‘, then under Dynamic content, select Output under Select Related Entities (previous step).

Select Alerts Name

The next step is to select the Alert Display Name. In the Logic app designer, under the step where you created an HTML table with related entities, select New step and then click Add an action.

Under Add an action, in the search box, type ‘Select‘ as your filter. From the actions list, choose Select under Data Operations.

Provide this information about your selection:

From

• For the value, select Insert Expression ‘fx‘, and then under Dynamic content, start typing ‘alerts‘ and select Alerts.

Map

• For the value, type Alert Name and then enter ‘item()?['properties']?['alertDisplayName']‘ under Function.

Create HTML table with Alerts Name

Once we have selected the alert display name, we will send the output to an HTML table.

In the Logic app designer, under the step where you added the select alerts name, select New step and then click Add an action.

Under Add an action, in the search box, type ‘HTML‘ as your filter. From the actions list, choose Create HTML table under Data Operations.

Provide this information about your selection:

From

• For the value, select Insert Expression ‘fx‘, then under Dynamic content, select Output under Select Alerts Name (previous step).

Extract Custom Details

We are now done extracting the incident entities, including all the entity mapping that is part of the Analytics Rules > Rule logic you defined.

The next step is to extract Custom Details from the incident. This will involve multiple steps because the Custom Details field is a JSON string that contains various entities. It is included under Alerts > Additional data of the incident body JSON. When this string is parsed, it becomes a JSON object containing several key-value pairs, where each key is associated with an array of values.

Get Alert Custom Details

In the Logic app designer, under the step Create HTML table with Alerts Name, select New step and then click Add an action.

Under Add an action, in the search box, type ‘Compose‘ as your filter. From the actions list, choose Compose under Data Operations.

Provide this information about your selection:

Inputs

• For the value, select Insert Expression ‘fx‘, and then under Dynamic content, start typing ‘alert custom‘ and select Alert Custom Details. This will create an array that includes the Get Alert Custom Details step.

Initialize an Array variable

In the Logic app designer, under the For each array step where you want to add a variable, select New step and then click Add an action.

Under Add an action, in the search box, type ‘variables‘ as your filter. From the actions list, select the Initialize variable.

Provide this information about your variable:

• For the variable name, use ‘AlertCustomDetails‘.
• For the type, select Array.
• For the value, select Insert Expression ‘fx‘, and then under Function, enter ‘outputs('Get_Alert_Custom_Details')‘ and then click Add.

Parse JSON Custom Details

In your logic app, under the Initialize variable step where you want to parse the Custom Details, select New step and then click Add an action.

Under Add an action, in the search box, type ‘Parse JSON‘ as your filter. From the actions list, select the Parse JSON under Data Operations.

Provide this information about your selection:

Content

• For the value, select Insert Expression ‘fx‘, and then under Dynamic content, choose the Alert Custom Details variable (previous step).

Schema

• We need to put the variable’s value in an array string for the schema. Use the following schema.

{
    "type": "array",
    "items": {
        "type": "string"
    }
}

Clean JSON Custom Details

In the next step, we need to iterate through each Custom Details item to remove unnecessary characters and then parse the output into a JSON string so we can later format it using an HTML table.

In your logic app, under the previous step, Parse JSON Custom Details, select New step, and click Add an action.

Under Add an action, in the search box, type ‘For each‘ as your filter. From the actions list, select the For each under Control.

For the value, select the output from the previous step as a Function: ‘body('Parse_JSON_Custom_Details')‘, as shown in the figure below.

Clean JSON String

Inside the For each loop block, insert a New step and click Add an action.

Under Add an action, in the search box, type ‘Compose‘ as your filter. From the actions list, choose Compose under Data Operations.

Provide this information about your selection:

Inputs

• For the value, select Insert Expression ‘fx‘, then under Function, enter the following compose action to remove unnecessary characters from all Custom Details values. We’ll remove all backslashes and newline characters, convert escaped double quotes to normal double quotes, and remove opening and closing square brackets.

replace(replace(replace(replace(replace(item(), '\\\\', ''), '\\n', ''), '\\\"', '\"'), '[', ''), ']', '')

Select an existing incident

Next, we must run the playbook once to capture the output and parse the Custom Details entities.

In Microsoft Sentinel, navigate to the Incidents page and select an incident containing Custom Details for which you want to run the playbook.

Select Actions > Run playbook on the incident page on the right.

Under Playbooks, next to the playbook you created, select Run. When the playbook is triggered, you’ll see a message on the top right indicating that the Playbook was triggered successfully, as shown in the figure below.

Next, select the Runs tab, and next to your playbook name, select View Run, as shown in the figure below.

The Logic app run page will open. The sample payload is visible under the Clean JSON String step > Outputs, as shown in the figure below. Note the sample payload Outputs for the next step, where we must parse the results into a JSON file.

Parse JSON String

In your logic app, inside the For each loop block, insert a New step under the Clean JSON String step and click Add an action.

Under Add an action, in the search box, type ‘Parse JSON‘ as your filter. From the actions list, select the Parse JSON under Data Operations.

Provide this information about your selection:

Content

• For the value, select Insert Expression ‘fx‘, then under Dynamic content, choose the Outputs for the Clean JSON String: ‘outputs('Clean_JSON_String')‘.

Schema

• Under Schema, paste a JSON schema to extract cleaned values from an array. Copy all the Outputs you generated when you ran the playbook from the previous step (Select an existing incident).
• Return to the playbook, select Use sample payload to generate schema, paste the entire payload (including the opening and closing square brackets), and select Done.

Here is an example of our schema type and properties (yours is going to be different).

Create HTML table for Custom Details

Once we have the Custom Details results clean, we will send the output to an HTML table.

In the Logic app designer, after the For Each Custom Details Item step, insert the New step and then click Add an action.

Under Add an action, in the search box, type ‘HTML‘ as your filter. From the actions list, choose Create HTML table under Data Operations.

Provide this information about your selection:

From

• For the value, select Insert Expression ‘fx‘, then under Function, select Output of the previous step Parse JSON String: ‘body('Parse_JSON_String')‘.

Advanced parameters

• Under the Advanced parameters, select Columns and then choose Automatic.

Test Playbook and Send Email

Once you have completed all the previous steps, you can use the values you created as dynamic content for further action. For example, if you want to send an email with process data that includes both Entity Mapping and Custom Details, you can do so by using the values you created as Dynamic content.

In the Logic app designer, after the Create HTML table for Custom Details step, insert the New step and then click Add an action.

Under Add an action, in the search box, type ‘Send email‘ as your filter. From the actions list, choose Send email under Azure Communication Email.

If you have provisioned the Azure Communication Service, you can use the three HTML tables we created in the previous steps as part of the email body if you haven’t changed the HTML action names:

• body('Create_HTML_table_with_Alerts_Name')
• body('Create_HTML_table_with_Related_Entities')
• body('Create_HTML_table_for_Custom_Details')

Lastly, ensure that the playbook is saved and that you can use it for SOC operations. Once an Automation rule triggers the playbook as an incident, you’ll receive an HTML email like the one below.

That’s it, there you have it… Happy alerting!

In Summary

leveraging Microsoft Sentinel’s custom details feature enhances security alerting and response capabilities. By including specific event data in your alerts, you streamline the investigation process for SOC analysts, enabling quicker and more effective threat response.

The step-by-step guide provided in this article demonstrates how to extract these custom details and integrate them into Logic Apps for automated responses. This integration ensures that critical information is promptly available, improving your organization’s overall security posture. By following these practices, you can maximize the efficiency of your incident response workflows and maintain robust security operations.

__
Thank you for reading our blog.

Please let us know in the comments section below if you have any questions or feedback.

-Charbel Nemnom-