How To Enable Immediate Sync On Azure File Share With Azure File Sync

7 min read

In this article, I will show you how to enable immediate Sync on Azure File Share with Azure File Sync.

Introduction

Azure File Sync can help you centralize your files in Azure and then install a sync agent on Windows Server whether it’s on-premises or in Azure (IaaS VM) to provide fast local access to your files. Your Windows Server and Azure Files are constantly in sync, so you have one centralized location for your files with multi-site access powered by fast local cache and cloud tiering.

The cloud tiering feature enables frequently accessed files to be cached locally such that the full file content is present on the server, whereas less frequently accessed files are tiered to the cloud. The tiered files (AFS reparse points) will be recalled on-demand when a user or application accesses it on the local server.

For more information about Azure File Sync, please make sure to check my previous articles.

Since the first release of Azure File Sync, one of the most requested features by many customers, was the ability to sync back when the changes occur directly on the Azure File Share. The user-voice is very active to improve this capability, you can add your vote here.

The good news is, Microsoft just added a new way to trigger sync that happens on files that are placed directly in the Azure File Share. Normally it can take up to 24 hours before sync will detect changes that happen directly in the Azure File Share. With this new Azure PowerShell cmdlet introduced by the Azure Storage Team, you can point sync to particular files, directories or sub-directories and have it look for changes, then sync back all the changes.

This new capability is intended for scenarios where some type of automated process in Azure is doing the file edits or migrations are done by an administrator (like moving a new directory of files into the file share). You can also use the same approach when a user adds or change files directly in Azure File Share, however, Microsoft recommends that for end-user direct changes on the Azure File Share, is to install Azure File Sync agent in an IaaS VM, and then give the end-user access through that. This way all changes will quickly sync to other agents without the need to call the Azure PowerShell cmdlet, but nothing stops you from using this method to tackle that scenario as well.

In this article, I will show you how to automate the immediate sync process and make sure that all the changes made on the Azure File Share will sync back to your Windows Server whether it’s on-premises or in Azure.

Create an Azure Automation Account

First, I need to create an Azure automation resource with Run As account. Run As accounts in Azure Automation are used to provide authentication for managing resources in Azure with the Azure cmdlets. When you create a Run As account, it creates a new service principal user in Azure Active Directory and assigns the Contributor role to this user at the subscription level.

Open the Azure portal, click All services found in the upper left-hand corner. In the list of resources, type Automation. As you begin typing, the list filters based on your input. Select Automation Accounts.

Click +Add. Enter the automation account name, choose the right subscription, resource group, location, and then click Create.

How To Enable Immediate Sync On Azure File Share With Azure File Sync 1

Import Modules from Gallery

In the next step, you need to import the required modules from the Modules gallery. In your list of Automation Accounts, select the account that you created in the previous step.

From your Automation Account, select Modules under Shared Resources. Click Browse Gallery to open the Browse Gallery page. You need to import the following modules from the Modules gallery in the order given below:

  1. Az.Accounts
  2. Az.StorageSync

How To Enable Immediate Sync On Azure File Share With Azure File Sync 2

Create PowerShell Runbook

In this step, you can create multiple Runbooks based on which set of Azure File Shares you want to sync back the changes. PowerShell Runbooks are based on Windows PowerShell. You directly edit the code of the Runbook using the text editor in the Azure portal. You can also use any offline text editor such as Visual Studio Code and import the Runbook into Azure Automation.

In this example, I will create a Runbook to detect and check the files and directories changes in a specific Sync Group Name, and in a specific Cloud Endpoint Name. You can also be creative as much as you want and cover multiple Azure File Shares / Sync Groups / Cloud Endpoints / Directories.

How To Enable Immediate Sync On Azure File Share With Azure File Sync 3

Edit The Runbook

Once you have the Runbook created, you need to edit the Runbook, then write or add the script to choose which Azure File Share you want to detect and sync back the changes. Of course, you can create scripts that suit your environment.

As mentioned earlier, in this example, I will create a Runbook to detect and check the files and directories changes in a specific Sync Group Name / Cloud Endpoint Name and in a specific directory.

The script as follows:

<#
.DESCRIPTION
A Runbook example which continuously check for files and directories changes in recursive mode
For a specific Azure File Share in a specific Sync Group / Cloud Endpoint
Using the Run As Account (Service Principal in Azure AD)

.NOTES
Filename : Enable-ImmediateFileSync
Author   : Charbel Nemnom
Version  : 1.0
Date     : 24-August-2019
Updated  : 26-August-2019

.LINK
To provide feedback or for further assistance please visit:
https://charbelnemnom.com 
#>

Param (
    [Parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]
    [String] $AzureSubscriptionId,
    [Parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]
    [String] $ResourceGroupName,
    [Parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]
    [String] $StorageSyncServiceName,
    [Parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]
    [String] $SyncGroupName,
    [Parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]
    [String] $Path
)

$connectionName = "AzureRunAsConnection"

Try {
    #! Get the connection "AzureRunAsConnection "
    $servicePrincipalConnection = Get-AutomationConnection -Name $connectionName
    Write-Output "Logging in to Azure..."
    Connect-AzAccount -ServicePrincipal `
        -TenantId $servicePrincipalConnection.TenantId `
        -ApplicationId $servicePrincipalConnection.ApplicationId `
        -CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint
}
Catch {
    If (!$servicePrincipalConnection) {
        $ErrorMessage = "Connection $connectionName not found..."
        throw $ErrorMessage
    }
    Else {
        Write-Error -Message $_.Exception
        throw $_.Exception
    }
}

Select-AzSubscription -SubscriptionId $AzureSubscriptionId

#! Get Cloud Endpoint Name
$azsync = Get-AzStorageSyncCloudEndpoint -ResourceGroupName "$ResourceGroupName" -StorageSyncServiceName $StorageSyncServiceName `
    -SyncGroupName "$SyncGroupName"
Write-Output "Get Azure StorageSync Cloud Endpoint Name: $($azsync.CloudEndpointName)"

#! Invoke-AzStorageSyncChangeDetection
Write-Output "Check for files and directories changes for $StorageSyncServiceName in $SyncGroupName"
Invoke-AzStorageSyncChangeDetection -ResourceGroupName $ResourceGroupName -StorageSyncServiceName $StorageSyncServiceName `
    -SyncGroupName $SyncGroupName -CloudEndpointName $azsync.CloudEndpointName -DirectoryPath $Path -Recursive

Write-Output ("")

Save the script in the CMDLETS pane as shown in the following screenshot.

How To Enable Immediate Sync On Azure File Share With Azure File Sync 4

Then test the script using “Test Pane” to verify it’s working as intended before you publish it. Once the test is completed, publish the Runbook by clicking Publish.

Schedule Azure File Sync Change Detection

In the final step, you need to schedule the Runbook to run base on your desired time to detect the Azure File Share changes.

Within the same Runbook that you create in the previous step, select Schedules and then click + Add schedule.

So, if you need to schedule the Runbook to run every hour, then you need to create the following schedule with Recur every 1 Hour with Set expiration to No. You can also run it on-demand if you wish to do so.

How To Enable Immediate Sync On Azure File Share With Azure File Sync 5

While scheduling the Runbook, you can pass on the parameters required for the PowerShell Script. In my example, I need to specify the Azure Subscription ID, Resource Group Name, Storage Sync Service Name, Sync Group Name, and the Directory Name that I want to detect the changes. The sample script takes those parameters as input.

How To Enable Immediate Sync On Azure File Share With Azure File Sync 6

Once done, click OK.

Test the Runbook

In this demo, I will test the Runbook by uploading a file directly to Azure File Share, and then I will request on-demand storage sync to detect the changes so the file will sync back to on-premises as well. This scenario will simulate when the user adds or change files directly in Azure File Share.

How To Enable Immediate Sync On Azure File Share With Azure File Sync 7

Monitor the Runbook

You can monitor the success or failure of these schedules using the “Jobs” tab of Runbooks under Resources. You can also see the next run schedule, in my example, the Runbook will run every hour, and so forth…

How To Enable Immediate Sync On Azure File Share With Azure File Sync 8

That’s it there you have it!

As a side note, running Invoke-AzStorageSyncChangeDetection will work for small data sets. Keep in mind that this will fail after 10K objects.

This is still version 1.0, if you have any feedback or changes that everyone should receive, please feel free to leave a comment below.

Summary

Azure File Sync extends on-premises file servers into Azure providing cloud benefits while maintaining performance and compatibility. Azure File Sync provides:

  • Multi-site access – provide write access to the same data across Windows servers and Azure Files.
  • Cloud tiering – store only recently accessed data on local servers.
  • Integrates with Azure backup – no need to back up your data on-premises.
  • Fast disaster recovery – restore file metadata immediately and recall data as needed.

I hope you find this guide useful. To learn more about Azure File Sync, please check the following articles.

__
Thank you for reading my blog.

If you have any questions or feedback, please leave a comment.

-Charbel Nemnom-

Related Posts

About Charbel Nemnom 579 Articles
Charbel Nemnom is a Cloud Architect, Swiss Certified ICT Security Expert, Microsoft Most Valuable Professional (MVP), and Microsoft Certified Trainer (MCT), totally fan of the latest's IT platform solutions, accomplished hands-on technical professional with over 17 years of broad IT Infrastructure experience serving on and guiding technical teams to optimize the performance of mission-critical enterprise systems. Excellent communicator is adept at identifying business needs and bridging the gap between functional groups and technology to foster targeted and innovative IT project development. Well respected by peers through demonstrating passion for technology and performance improvement. Extensive practical knowledge of complex systems builds, network design, business continuity, and cloud security.

Be the first to comment

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.