Manage Security Content as Code with Microsoft Sentinel

7 Min. Read

During Microsoft Ignite in November 2021, Azure Sentinel is now called Microsoft Sentinel.

Microsoft Sentinel is a cloud-native Security Information Event Management (SIEM) and Security Orchestration Automated Response (SOAR) solution. Microsoft Sentinel delivers intelligent security analytics and threat intelligence across the enterprise, providing a single solution for alert detection, threat visibility, proactive hunting, and threat response.

In this article, we will share with you how to manage security content as Code with Microsoft Sentinel and show you how to deploy analytics rules from Azure DevOps.

Introduction

During Microsoft Ignite in November 2021, Microsoft announced Microsoft Sentinel Repositories (public preview), a new capability that allows users to manage their Microsoft Sentinel content as code from a source control repository (GitHub or Azure DevOps). Repositories provide a central experience for deployment and management of Microsoft Sentinel content and remove the burden of having to manage manual processes to update and deploy your custom content across your Log Analytics workspaces.

The Microsoft Sentinel Repositories allow customers and partners to manage Sentinel as code and automate the deployment of their security content. It’s the same way that IT and cloud departments use things like Terraform, ARM Templates, or Bicep to define their VM’s and networking, etc. to deploy content. You can manage your security content that lives within Sentinel using this new capability.

This new capability is great for any company that wants to automate their content management in Sentinel because today it’s just very difficult to keep track of the changes that security teams are making to all the different artifacts that exist in Sentinel like Analytic rules, Parsers, Workbooks, Playbooks, Automation rules, and Hunting queries, so it’s a great way to automate the deployment across many different workspaces.

This also benefits Managed Security Service Providers (MSSP) who have to manage dozens of customers in parallel or different Log Analytics workspaces. Automation is key for those scenarios and everything that has to do with content security management and even more.

Prerequisites

To follow this article, you need to have the following:

1) Azure subscription – If you don’t have an Azure subscription, you can create a free one here.

2) Log Analytics workspace – To create a new workspace, follow the instructions to create a Log Analytics workspace.

3) Microsoft Sentinel – To enable Microsoft Sentinel at no additional cost on an Azure Monitor Log Analytics workspace for the first 31-days, follow the instructions here.

4) Azure DevOps – If you don’t have one, you can create a free one here.

5) Azure DevOps Organization/Project – If you don’t have one, follow these steps to create an organization.

6) Azure DevOps Git repository – If you don’t have one, follow these steps to create a new Git repository in your project.

7) An Owner role in the resource group that contains your Microsoft Sentinel workspace.

8) You need to have access to a GitHub or Azure DevOps repository, with any custom security content files you want to deploy to your workspaces, in relevant Azure Resource Manager (ARM) JSON templates (more on this in the next section).

9) Connecting Microsoft Sentinel workspace to an external source control repository (more on this in the next section).

At the time of this writing, Microsoft Sentinel currently supports connections only with GitHub and Azure DevOps repositories. For the remainder of this article, we will use Azure DevOps as a connected source control repository.

Azure DevOps Repository

We have already created an Azure DevOps repository dedicated to Microsoft Sentinel, and then we structured the repo based on all the different artifacts as shown in the figure below.

Azure DevOps Sentinel Repository
Azure DevOps Sentinel Repository

Next, we have cloned the repository and started authoring analytics (detection) rules using Visual Studio Code and pushed them to Azure DevOps as shown in the figure below.

Azure DevOps Analytics Rules
Azure DevOps Analytics Rules

For the remainder of this article, we will use the Analytics rules as an example to deploy security content as code with Microsoft Sentinel. The same can be applied to other artifacts as well, like Parsers, Workbooks, Playbooks, Automation rules, and Hunting queries.

Now switching to Microsoft Sentinel workspace, and on the left under Configuration, select the active Analytics rules as shown in the figure below. We don’t see any custom detection rules that are stored in my Azure DevOps repository. In this example, we have authored 9 analytics rules for Dynamics 365 (D365).

Microsoft Sentinel | Analytics rules
Microsoft Sentinel | Analytics rules

Connect a repository to Microsoft Sentinel

This section describes how to connect an Azure DevOps repository to Microsoft Sentinel workspace, where you can save and manage your custom security content, instead of using the Microsoft Sentinel portal. The same steps will apply if you connect a GitHub repository.

Before you start connecting a repository, make sure that you’re signed in to your source control app with the credentials you want to use for your connection. If you’re currently signed in using different credentials, sign out first.

Launch Microsoft Sentinel, and on the left under Content management, select Repositories and then select “+ Add new” as shown in the figure below.

Microsoft Sentinel | Repositories
Microsoft Sentinel | Repositories

On the Create a new connection page, enter a meaningful name and description for your connection.

> Name: Azure DevOps Analytics Rules.

> Description: This connection will deploy analytics rules from the Azure DevOps repository.

From the Source Control dropdown, select the type of repository you want to connect to (GitHub or Azure DevOps), and then select Authorize as shown in the figure below.

Create a new repository connection
Create a new repository connection

You’ll be automatically authorized to Azure DevOps using your current Azure credentials. To ensure valid connectivity, verify that you’ve authorized to the same Azure DevOps account that you’re connecting to from Microsoft Sentinel or use an InPrivate (Incognito) browser window to create your connection.

After the connection is authorized, from the dropdown lists that appear below, select your Azure DevOps Organization, Project, Repository, Branch, and Content Types as shown in the figure below. In this example, we’re deploying just the “Analytic rules” content type.

Manage Security Content as Code with Microsoft Sentinel 1

> Both “Parsers” and “Hunting queries” use the Saved Searches API to deploy content to Microsoft Sentinel. If you select one of these content types and also have content of the other type in your branch, both content types are deployed.

> However, for all other content types such as Analytic rules, Workbooks, Playbooks, and Automation rules deploys only that content to Microsoft Sentinel. Content of other types is not deployed.

Last, select the Create button to create your connection.

Please note that you cannot create duplicate connections, with the same repository and branch, in a single Microsoft Sentinel workspace.

Verify Microsoft Sentinel deployment

After the connection is created as described in the previous section, a new workflow or pipeline is generated in your Azure DevOps/GitHub repository, and the content stored in your repository is deployed to your Microsoft Sentinel workspace.

The deployment time may vary depending on the amount of security content that you’re deploying.

The connection details on the Repositories page are updated with the link to the connection’s deployment logs. You will see the Name, Description, and the Repository as shown in the figure below

Source Control details
Source Control details

One most important point that you want to be aware of is the Deployment logs link. You could simply just click on this deployment logs link and it’ll take you directly to the Azure Pipelines page where this deployment is happening.

Deployment logs
Deployment logs

So this is the pipeline definition that was created for this deployment, the job goes into the queue, and it goes into a few minutes of deployment as shown in the figure below.

Azure DevOps Pipeline definition
Azure DevOps Pipeline definition

After the deployment is completed, the content stored in your repository is deployed in your Microsoft Sentinel workspace, on the relevant Microsoft Sentinel page. In this example, the Analytics rules. We can see now that we have 9 analytics custom rules for Dynamics 365 (D365) deployed as shown in the figure below.

Analytics rules deployed
Analytics rules deployed

Edit and Customize Pipeline deployment

By default, the content deployment deploys all of the relevant custom content from the connected repository branch whenever a push is made to anything in that branch.

If the default configuration for your content deployment from GitHub or Azure DevOps does not meet all your requirements, then you can modify the experience to fit your needs.

For example, you may want to configure different deployment triggers or deploy content only from a specific root folder. You may also want to split the SIEM and SOAR services into different Azure resource groups. The reason for that is, you can handle better the Role-based access control (RBAC) permissions and have a better overview of the resources that you have.

For more information about customizing the deployment workflow, please check Microsoft Sentinel documentation.

Manage Security Content Best Practices

After you’ve successfully created a connection to your source control repository (GitHub or Azure DevOps), anytime that content in that repository is modified or added, the deployment workflow runs again and deploys all content in the repository to all connected Microsoft Sentinel workspaces.

Please note that it’s highly recommended to edit any content stored in a connected repository only in the repository, and not in the Microsoft Sentinel portal (UI). For example, to make changes to your analytics rules as described in this article, do so directly in Azure DevOps or GitHub.

If you have edited the content in Microsoft Sentinel, then make sure to export it to your source control repository to prevent your changes from being overwritten the next time the repository content is deployed to your workspace.

And if you are deleting content, make sure to delete it from both your repository and the Azure Portal. Because deleting content from your repository does not delete it from your Microsoft Sentinel workspace.

We normally recommend to customers that if they are using a DevOps tool like Azure DevOps and GitHub to deploy security content into the workspace, it’s important to work on a good naming convention. For example, you can always put a prefix in the rules that say DevOps, or CI-CD, etc. So the user can identify easily that that rule is being managed by code and you shouldn’t be making changes to the UI because they might be lost.

It’s very important to have a clean way to manage all those rules and all that content you deploy from the repository because otherwise, it could be a bit of a management nightmare.

It’s also recommended to test the content in your repository on a development workspace through a connection there, before merging it into the branch that is connected to your main/prod workspaces. And the good news is, you can connect many workspaces to a single repository and branch. No limit on that. You can also connect a single workspace, to multiple repositories.

That’s it there you have it!

Summary

In this article, we showed you how to manage security content as Code with Microsoft Sentinel and how to deploy analytics rules from the Azure DevOps repository.

Microsoft Sentinel Repositories helps you automate the deployment and management of your Microsoft Sentinel content through central repositories.

You connect GitHub or Azure DevOps content repositories by selecting ‘Add new’ and following the steps as described in this article. Creating a connection will create a workflow in your source control content repository. This workflow is responsible for automatically deploying your Microsoft Sentinel content into your connected workspaces anytime content is modified or added to the repository.

At the time of this writing, Microsoft Sentinel Repositories is in public preview and you can expect a lot of improvements when it hits general availability.

For more information, please check Microsoft Sentinel content and solutions documentation.

__
Thank you for reading my blog.

If you have any questions or feedback, please leave a comment.

-Charbel Nemnom-

Related Posts

Previous

Sync and Backup Synology NAS to Azure Storage

This Holiday Season, Win with Hornetsecurity!

Next

Let me know what you think, or ask a question...

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Subscribe to Stay in Touch

Never miss out on your favorite posts and our latest announcements!

The content of this website is copyrighted from being plagiarized!

You can copy from the 'Code Blocks' in 'Black' by selecting the Code.

Please send your feedback to the author using this form for any 'Code' you like.

Thank you for visiting!