How to Set up Azure File Sync #Azure #AzureFiles #AzureFileSync #WindowsServer

Updated – 01/04/2022 – Azure File Sync agent version 15 has started.

Updated – 03/12/2021 – Azure File Sync agent version 14.1 released.

Updated – 13/07/2021 – Azure File Sync agent version 13 released.

Updated – 21/06/2021 – Azure File Sync is now GA in China East 2 and China North 2! This is a sovereign offering, meaning it is an independent, dedicated network within China.

Updated – 31/05/2021 – Azure File Sync agent version 12.1 released.

Updated – 30/03/2021 – Azure File Sync agent version 12 released.

Updated – 08/02/2021 – Retiring AzureRM local Azure File Sync PowerShell module.

Azure File Sync helps you save on storage capacity by centralizing your files in Azure and then installing a sync agent on Windows Server on-premises or in Azure (IaaS VM) to provide fast local access to your cloud files. Your local server and Azure are constantly syncing, so you have one centralized location for your files with multi-site access powered by fast local caches and cloud tiering.

In this article, I will show you how to set up Azure File Sync so you get started and extend on-premises file servers to the cloud.

Introduction

You have a lot of files, and all are pilling up in scattered branch offices, that are difficult to manage, backup, and recover. You can consolidate your files in the cloud, but on-premises users and applications might struggle with the slower access times. At Ignite 2017 in September, Microsoft announced the public preview for Azure File Sync (AFS) which can help you to extend your on-premises file servers to Azure. In less than a year, Microsoft today made Azure File Sync (AFS) generally available.

Azure File Sync was built based on the following common problems:

> On-premises hardware management is a major pain both in terms of hardware being very expensive, but also in terms of managing disks and storage.

> Capacity management was another problem, and most IT admins want to keep everything on their file servers without deleting any data.

> Managing global infrastructure, and managing projects that span multiple sites is challenging and very complex because global access is just a fact of day-to-day life.

What is Azure File Sync?

With Azure File Sync, you can centralize your files in Azure and then install the Azure File Sync agent on a Windows Server on-premises to provide fast local access to your cloud files. Your local server and Azure are constantly syncing, so you have one centralized location for your files with multi-site access powered by fast local caches and cloud tiering.

What Cloud Tiering does is, over time builds up a heat map on your disks of what files are being used, what files are being written to, and then as the disks become full, the files will be moved to the cloud and keep only stubs (namespace) on the disks locally, and when the user clicks on the file, it will download seamlessly from the cloud rather than opening straight from disks. And this is desirable for those files that you are not using very often but still, you want them to be available.

If a server becomes unavailable on-premises for any reason, you just need to install Azure File Sync on another server or virtual machine, and your users and applications can access the file again within minutes. With the integration of the Azure Backup service, you can protect your data in the cloud without worrying about on-premises backup solutions. And if you want to move an application to the cloud, your App can access the Azure File share just like a local file share in your network. With Azure File Sync, you get the benefits of consolidating your files in the cloud, matched with the performance of a local file server.

Azure File Sync is a hybrid solution that is built on top of the Azure Platform as a Service (PaaS) and Azure Files.

In this article, I will show you how to set up Azure File Sync so you get started and extend on-premises file servers to the cloud.

Prerequisites for Azure File Sync

Before we get started, there are a few prerequisites that you need to verify.

On-premises

The supported versions of Windows Server by Azure File Sync are:

• Windows Server 2022 (Datacenter and Standard) – Server with Desktop Experience
• Windows Server 2019 (Datacenter and Standard) – Server with Desktop Experience
• Windows Server 2016 (Datacenter and Standard) – Server with Desktop Experience
• Windows Server 2012 R2 (Datacenter and Standard) – Server with Desktop Experience

Updated: Microsoft confirmed that Windows Server Core SKU is supported with Azure File Sync as well. Please check the following article to learn more on how to install Azure File Sync on Windows Server Core.

PowerShell version 5.1 is the minimum required for Azure File Sync. If you are using Windows Server 2012 R2, then you need to ensure to install at least PowerShell 5.1. You can safely skip this check on Windows Server 2016, Windows Server 2019, and Windows Server 2022.

On Azure

At the time of this writing, Azure File Sync is only available in 30 regions. Microsoft will continue to add support for more regions soon. You can check region availability here.

• Australia East
• Brazil South
• Australia Southeast
• Canada Central
• Canada East
• Central India
• Central US
• East Asia
• East US
• East US2
• France Central
• Korea Central
• Korea South
• Japan East
• Japan West
• North Europe
• North Central US
• Southeast Asia
• South Central US
• South India
• South Africa North
• South Africa West (*)
• UAE Central
• UAE North (*)
• UK South
• UK West
• West Europe
• West US
• West Central US
• West US 2
• China East 2
• China North 2

For the regions marked with asterisks (*), you must contact Azure Support to request access to Azure Storage in those regions. Last, you need to create a storage account in Azure, and then create a file share on top of that storage account.

For a full list of Azure File Sync availability, please check the product availability by region here.

Deploy Azure File Sync

In the following section, I will describe the required steps to automate and deploy Azure File Sync with PowerShell. The same steps can be accomplished through the UI and the Azure Portal.

Step 1 – Prepare Windows Server to use with Azure File Sync

Open an elevated PowerShell console and run the following command to disable the Internet Explorer Enhanced Security Configuration for Administrators and Users. This is required only for initial server registration in Step 5.

# Disable Internet Explorer Enhanced Security Configuration for Administrators
Set-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Active Setup\Installed Components\{A509B1A7-37EF-4b3f-8CFC-4F3A74704073}" -Name "IsInstalled" -Value 0 -Force

# Disable Internet Explorer Enhanced Security Configuration for Users
Set-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Active Setup\Installed Components\{A509B1A8-37EF-4b3f-8CFC-4F3A74704073}" -Name "IsInstalled" -Value 0 -Force

Once you register your server with Azure File Sync, you can re-enable Internet Explorer Enhanced Security Configuration by updating the Value from 0 to 1.

Step 2 – Download Azure File Sync Agent

The next step is to download the Azure File Sync agent. Open an elevated PowerShell console on a Windows server and run the following command. This will download the appropriate Azure File Sync agent based on your Operating System version (Windows Server 2012 R2, Windows Server 2016, and Windows Server 2019).

# Gather the OS version
$osver = [System.Environment]::OSVersion.Version

# Download the appropriate version of the Azure File Sync agent for your OS.
# Azure File Sync Version 15.0
# Windows Server 2022
if ($osver.Equals([System.Version]::new(10, 0, 20348, 0))) {
    Invoke-WebRequest `
        -Uri https://download.microsoft.com/download/1/8/D/18DC8184-E7E2-45EF-823F-F8A36B9FF240/StorageSyncAgent_WS2022.msi `
        -OutFile "StorageSyncAgent.msi" 
}
# Windows Server 2019
elseif ($osver.Equals([System.Version]::new(10, 0, 17763, 0))) {
    Invoke-WebRequest `
        -Uri https://download.microsoft.com/download/1/8/D/18DC8184-E7E2-45EF-823F-F8A36B9FF240/StorageSyncAgent_WS2019.msi `
        -OutFile "StorageSyncAgent.msi" 
}
# Windows Server 2016
elseif ($osver.Equals([System.Version]::new(10, 0, 14393, 0))) {
    Invoke-WebRequest `
        -Uri https://download.microsoft.com/download/1/8/D/18DC8184-E7E2-45EF-823F-F8A36B9FF240/StorageSyncAgent_WS2016.msi `
        -OutFile "StorageSyncAgent.msi" 
}
# Windows Server 2012 R2
elseif ($osver.Equals([System.Version]::new(6, 3, 9600, 0))) {
    Invoke-WebRequest `
        -Uri https://download.microsoft.com/download/1/8/D/18DC8184-E7E2-45EF-823F-F8A36B9FF240/StorageSyncAgent_WS2012R2.msi `
        -OutFile "StorageSyncAgent.msi" 
}
else {
    throw [System.PlatformNotSupportedException]::new("Azure File Sync is only supported on Windows Server 2012 R2, Windows Server 2016, Windows Server 2019, and Windows Server 2022")
}

Azure File Sync updates

In April 2022, Microsoft released Azure File Sync (AFS) Agent v15 in a flighting mode to servers that are configured to automatically update when a new version becomes available. The AFS agent 15 addresses the following issues and improvements:

• Reduced transactions when cloud change enumeration job runs.
• View Cloud Tiering status for a server endpoint or volume.
• New diagnostic and troubleshooting tool.
• Immediately run server change enumeration to detect file changes that were missed by the USN journal.
• Reliability and telemetry improvements for cloud tiering and sync.
• This release is available for Windows Server 2012 R2, Windows Server 2016, Windows Server 2019, and Windows Server 2022 installations.

To obtain and install the v15 update, you configure your Azure File Sync agent to automatically update when a new version becomes available or you can manually download the update from the Microsoft Update Catalog. The installation instructions for v15 are documented in the following KB5003882.

I highly encourage you to upgrade to version 15 of the Azure File Sync agent to reduce storage transaction costs and save on your bill.

In December 2021, Microsoft released Azure File Sync (AFS) Agent v14.1 in a flighting mode to servers that are configured to automatically update when a new version becomes available. The AFS agent 14.1 addresses the following issues and improvements:

• Improved server endpoint de-provisioning guidance in the Azure Portal.
• Invoke-AzStorageSyncChangeDetection cmdlet improvements. Check the following step-by-step guide for more details.
• Azure File Sync is now supported in the West US 3 region.
• Reduced transactions when a file consistently fails to upload due to a per-item sync error.
• Support for Windows Server 2022 GA.
• Reduced transactions when cloud change enumeration job runs.
• Reliability and telemetry improvements for cloud tiering and sync.
• Issue fixed: Tiered files deleted on Windows Server 2022 are not detected by the cloud tiering filter driver.

To obtain and install the v14.1 update, you can configure your Azure File Sync agent to automatically update when a new version becomes available or you can manually download the update from the Microsoft Update Catalog. The installation instructions for v14.1 are documented in the following KB5001873.

In July 2021, Microsoft released Azure File Sync (AFS) Agent v13 in a flighting mode to servers that are configured to automatically update when a new version becomes available. The AFS agent 13 addresses the following issues and improvements:

• Authoritative upload
• Portal improvements to viewing cloud change enumeration and sync progress
• Support for server rename
• Support for Windows Server 2022 Preview. Note: Windows Server 2022 adds support for TLS 1.3 which is not currently supported by Azure File Sync. If the TLS settings are managed via group policy, the server must be configured to support TLS 1.2.
• Miscellaneous improvements

To obtain and install the v13 update, you can configure your Azure File Sync agent to automatically update when a new version becomes available or you can manually download the update from the Microsoft Update Catalog here. The installation instructions for v13 are documented in the following KB4588753.

In May 2021, Microsoft released v12.1 directly to Microsoft Update and Microsoft Download Center because it contains a fix for the v12 agent auto-update issue. Improvements and issues that are fixed:

• The v12.0 agent release had two bugs which are fixed in this release:
• Agent auto-update fails to update the agent to a later version.
• FileSyncErrorsReport.ps1 script does not provide the list of per-item errors.

Note: If agent version 12.0 is installed on your servers, you will need to update to v12.1 using Microsoft Update or Microsoft Update Catalog here.

More information about the version 12.1 release:

• This update is available for Windows Server 2012 R2, Windows Server 2016, and Windows Server 2019 installations.
• The agent version for this release is 12.1.0.0.
• A restart may be required if files are in use during the installation.
• Installation instructions are documented in KB4588751.

In March 2021, Microsoft released Azure File Sync (AFS) Agent v12 in a flighting mode to servers that are configured to automatically update when a new version becomes available. The AFS agent 12 addresses the following issues and improvements:

• New portal experience to configure network access policy and private endpoint connections
• Cloud Tiering support for volume cluster sizes larger than 64KiB
• Measure bandwidth and latency to Azure File Sync service and storage account
• Improved error messages in the portal when server endpoint creation fails
• Miscellaneous performance and reliability improvements

To obtain and install the v12 update, you can configure your Azure File Sync agent to automatically update when a new version becomes available. The installation instructions for v12 are documented in the following KB4568585.

In February 2021, Microsoft released Azure File Sync (AFS) Agent v11.2 in a flighting mode to servers that are configured to automatically update when a new version becomes available. The AFS agent 11.2 addresses the following issues and improvements:

• If a sync session is canceled due to a high number of per-item errors, sync may go through reconciliation when a new session starts if the Azure File Sync service determines a custom sync session is needed to correct the per-item errors.
• Registering a server using the Register-AzStorageSyncServer cmdlet may fail with an “Unhandled Exception” error.
• New PowerShell cmdlet (Add-StorageSyncAllowedServerEndpointPath) to configure allowed server endpoints paths on a server. This PowerShell cmdlet is useful for scenarios where a Cloud Solution Provider (CSP) or Service Provider manages the Azure File Sync deployment, and you want to configure the allowed server endpoints paths on a server. Please note that when creating a server endpoint, if the path specified is not in the allow list, the server endpoint creation will fail. Note: this is an optional feature and all supported paths are allowed by default when creating a server endpoint.

To obtain and install the v11.2 update, you can configure your Azure File Sync agent to automatically update when a new version becomes available or you can manually download the update from the Microsoft Update Catalog here. The installation instructions for v11.2 are documented in the following KB4539952.

Step 3 – Install Azure File Sync agent

The next step is to install the Azure File Sync agent. Open an elevated PowerShell console on your Windows server and run the following command:

# Install the MSI package in quiet mode
Start-Process -FilePath ".\StorageSyncAgent.msi" -ArgumentList "/quiet" -Wait

The Azure File Sync agent consists of three main components:

> FileSyncSvc.exe: The background Windows service responsible for monitoring changes on Server Endpoints and initiating sync sessions to Azure.

> StorageSync.sys: The Azure File Sync file system filter, is responsible for tiering cold files to Azure Files (when cloud tiering is enabled).

> PowerShell management cmdlets: PowerShell cmdlets for interacting with Microsoft.StorageSync Azure Resource Provider. The cmdlets can be found at the following locations (by default):

• %ProgramFiles%\Azure\StorageSyncAgent\StorageSync.Management.ServerCmdlets.dll
• %ProgramFiles%\Azure\StorageSyncAgent\StorageSync.Management.PowerShell.Cmdlets.dll

Retiring AzureRM local Azure File Sync PowerShell module

Please note that Microsoft stopped investing in updating the local Azure File Sync PowerShell module which is installed by default under:

%ProgramFiles%\Azure\StorageSyncAgent\StorageSync.Management.PowerShell.Cmdlets.dll

This article has been updated to use the public Azure Management PowerShell module “Az module”.

You can use the following PowerShell command to install and update to the latest “Az module” on each server you intend to use with Azure File Sync.

# Make sure you have the latest version of PowerShellGet installed
Install-Module -Name PowerShellGet -Force
# Install and update to the latest Az PowerShell module
Install-Module -Name Az -AllowClobber -Force

Please check your scripts to make sure your management tasks won’t start breaking with the next Azure File Sync agent release.

Step 4 – Deploy the Storage Sync Service in Azure

The next step is to create the Storage Sync Service in Azure.

Open an elevated PowerShell console on your Windows server and run the following script:

# Login to Azure
Try {
    $AzureSubName = Read-Host "`n Enter Azure Subscription name"
    $AzAcctInfo = Login-AzAccount -Subscription $AzureSubName
    Write-Output "Connecting to Azure Cloud..."
   }
   Catch {
     Write-Output "Cannot connect to Azure environment. Please check your credentials and Azure Subscription name. Exiting!"
     Break
}

# The default location of the Azure File Sync Agent
$AFSAgentPath = "C:\Program Files\Azure\StorageSyncAgent"

# Enter any of the supported Azure regions you want to deploy Azure File Sync into
# https://docs.microsoft.com/en-us/azure/storage/files/storage-sync-files-planning#region-availability
$AzRegion = Read-Host "`n Enter Azure Region name"

# Enter the resource group name to deploy the Storage Sync Service into
$AzResourceGroup = Read-Host "`n Enter Azure Resource Group name"
New-AzResourceGroup -Name $AzResourceGroup -Location $AzRegion

# Create Azure Storage Sync Service
$AFSStorageSyncName = Read-Host "`n Enter the desired name of your Storage Sync Service"
$storageSync = New-AzStorageSyncService -ResourceGroupName "$AzResourceGroup" -Name "$AFSStorageSyncName" -Location "$AzRegion"

Do NOT close the PowerShell session after running the script above, this will help to run subsequent Azure File Sync PowerShell commands with minimal repetition of parameters and authentication.

Step 5 – Register Windows Server with Storage Sync Service

In this step, you need to register your server on-premises with Azure Storage Sync Service. Open an elevated PowerShell console on your Windows server and run the following command:

# Register Windows Server with the Sync Service
RegisteredServer = Register-AzStorageSyncServer -StorageSyncServiceName "$AFSStorageSyncName" -ResourceGroupName "$AzResourceGroup"

Registering your Windows Server with a Storage Sync Service establishes a trust relationship between your server on-premises and the Storage Sync Service in Azure. If you are deploying Azure File Sync on a node that is a member of a Failover Cluster, then you need to repeat the same steps described above on each server node in a Failover Cluster.

Step 6 – Create a Storage account and file share

As mentioned earlier in this article, we need to create a storage account and file share in Azure before we start syncing. Open an elevated PowerShell console and run the following script:

# Create a storage account with desired name
$AzStorageAccountName = Read-Host "`n Enter the desired storage account name in lowercase letters only"

$AzStorageAccount = New-AzStorageAccount `
        -Name $AzStorageAccountName `
        -ResourceGroupName $AzresourceGroup `
        -Location $Azregion `
        -SkuName Standard_LRS `
        -Kind StorageV2 `
        -EnableHttpsTrafficOnly:$true

# Create an Azure file share within the desired storage account
$FileShareName = Read-Host "`n Enter the desired Azure file share name in lowercase letters only"
$FileShare = New-AzStorageShare -Context $AzStorageAccount.Context -Name $fileShareName

Step 7 – Create a Sync Group

In this step, we need to create a sync group. A sync group is the basic topology unit of Azure File Sync, it defines the relationship of a sync partnership.

Open an elevated PowerShell console and run the following command:

$SyncGroupName = Read-Host "`n Enter the desired Sync Group Name"
New-AzStorageSyncGroup -ResourceGroupName "$AzResourceGroup" -SyncGroupName "$SyncGroupName" -StorageSyncService "$AFSStorageSyncName"

Step 8 – Create a cloud endpoint

In this step, we need to create a cloud endpoint. A cloud endpoint is a pointer to an Azure file share. All server endpoints (Step 9) will sync with a cloud endpoint, making the cloud endpoint the hub.

Open an elevated PowerShell console and run the following command:

# Create a cloud endpoint
New-AzStorageSyncCloudEndpoint `
    -Name "CloudEndpoint-$SyncGroupName" `
    -ResourceGroupName "$AzResourceGroup" ` 
    -StorageSyncServiceName $AFSStorageSyncName `
    -SyncGroupName $SyncGroupName `
    -StorageAccountResourceId $AzStorageAccount.Id `
    -StorageAccountShareName $fileShare.Name

Step 9 – Create a server endpoint

In the last step, we need to create a server endpoint. A server endpoint represents a path on a registered server on disk on-premises, such as a folder on a server volume. The path must be on a non-system volume if you plan to enable Cloud Tiering. Cloud Tiering cannot be enabled on the system volume.

Open an elevated PowerShell console on the registered server and run the following command:

$ServerEndpointPath = Read-Host "`n Enter the desired path on your registered server which is not on the system volume i.e. D:\Data"

$DriveLetter = $ServerEndpointPath.Substring(0,1)
$volume = Get-Volume -DriveLetter $DriveLetter
$volumeFreeSpace = “{0:P0}” –F ($volume.SizeRemaining/$volume.Size)
$volumeFreeSpace = $volumeFreeSpace.Substring(0,2)
$volumeFreeSpacePercentage=[int]$volumeFreeSpace

# Get all registered server endpoints
$RegisteredServer = Get-AzStorageSyncServer -ResourceGroupName $AzResourceGroup `
    -StorageSyncServiceName $AFSStorageSyncName

# Create server endpoint
New-AzStorageSyncServerEndpoint `
    -Name $RegisteredServer[0].FriendlyName `
    -ResourceGroupName "$AzResourceGroup" `
    -StorageSyncServiceName $AFSStorageSyncName `
    -SyncGroupName $syncGroupName `
    -ServerResourceId $registeredServer[0].ResourceId  `
    -ServerLocalPath $serverEndpointPath `
    -CloudTiering `
    -VolumeFreeSpacePercent $volumeFreeSpacePercentage

Testing Azure File Sync

Now you have Azure File Sync deployed and ready to use. You can copy some files to the server endpoint under the path specified in Step 9. Based on your Internet bandwidth and the size of the files, storage sync will automatically kick off behind the scene and the files will be synced to Azure File share as shown in the following screenshot.

Since I enabled cloud tiering and I set the volume of free space percentage to a maximum of 100%.

All the files have been tiered to Azure file share. The following screenshot shows the actual size of the file and the size locally on disk.

That’s it there you have it!

Summary

Azure File Sync extends on-premises file servers into Azure providing cloud benefits while maintaining performance and compatibility.

Azure File Sync provides:

• Multi-site access – provide write access to the same data across Windows servers and Azure Files.
• Cloud tiering – store only recently accessed data on local servers.
• Integrates with Azure backup – no need to back up your data on-premises.
• Fast disaster recovery – restore file metadata immediately and recall data as needed.

I hope you find this article useful.

__
Thank you for reading my blog.

If you have any questions or feedback, please leave a comment.

-Charbel Nemnom-