Azure File Sync is Now Generally Available (GA) #Azure #AzureFiles #AzureFileSync #WindowsServer #AFS

12 min read

Updated – 21/06/2019 – Azure File Sync Agent v7.0 is released (KB4490495).

Updated – 13/06/2019 – Azure File Sync Agent v6.2 is released (KB4489738).

Updated – 07/06/2019 – Azure File Sync is now GA for Azure Government cloud.

Updated – 07/05/2019 – Azure File Sync Agent v6.1 is released (KB4489737).

Updated – 01/05/2019 – Azure File Sync support on Windows Server Core.

Updated – 23/04/2019 – Azure File Sync Agent v6 is now available (KB4489736).

Updated – 05/04/2019 – Azure File Sync Agent v5.2 is released (KB4481061).

Updated – 08/03/2019 – Azure File Sync Agent v5.1 is released (KB4481060).

Updated – 02/12/2019 – Azure File Sync Agent v5.0 is released (KB4459989).

Updated – 01/14/2019 – Azure File Sync Agent v4.3 is released (KB4481059).

Introduction

You have a lot of files, and all are pilling up in scattered branch offices, that are difficult to manage, backup and recover. You can consolidate your files in the cloud, but on-premises users and applications might struggle with the slower access times. At Ignite 2017 in September last year, Microsoft announced the public preview for Azure File Sync (AFS) which can help you to extend your on-premises file servers to Azure. In less than a year, Microsoft today made Azure File Sync (AFS) generally available.

Azure File Sync was built based on the following common problems:

  • On-premises hardware management is a major pain both in terms of hardware being very expensive, but also in terms of managing disks and storage.
  • Capacity management was another problem, and most IT admins want to keep everything on their file servers without deleting any data.
  • Managing global infrastructure, and managing projects that span multiple sites is challenging and very complex, because the global access is just a fact of day to day life.

What is Azure File Sync

With Azure File Sync, you can now centralize your files in Azure and then install Azure File Sync agent on a Windows Server on-premises to provide fast local access to your cloud files. Your local server and Azure are constantly syncing, so you have one centralized location for your files with multi-site access powered by fast local caches and cloud tiering. What Cloud Tiering does is, it over time builds up a heat map on your disks of what files are being used, what files are being written to, and then as the disks become full, the files will be moved to the cloud and keep only stubs (namespace) on the disks locally, and when the user clicks on the file, it will download seamlessly from the cloud rather than opening straight from disks. And this is obliviously desirable for those files that you are not using very often but you still want them to be around.

If a server becomes unavailable on-premises for any reason, you just need to install Azure File Sync on another server or virtual machine, and your users and applications can access the file again within minutes. With the integration of Azure Backup service, you can protect your data in the cloud without worrying about on-premises backup solutions. And if you want to move an application to the cloud, your App can access the Azure File share just like a local file share in your network. With Azure File Sync, you get the benefits of consolidating your files in the cloud, matched with the performance of a local file server.

Azure File Sync is a hybrid solution that is built on top of the Azure Platform as a Service (PaaS) and Azure Files.

In this blog post, I will show you how to quickly get started with Azure File Sync (AFS) to extend on-premises file servers to the cloud.

Prerequisites for Azure File Sync

Before we get started, there are few prerequisites that you need to verify.

On-premises

As of this writing, the supported versions of Windows Server by Azure File Sync are:

  • Windows Server 2019 (Datacenter and Standard) – Server with Desktop Experience
  • Windows Server 2016 (Datacenter and Standard) – Server with Desktop Experience
  • Windows Server 2012 R2 (Datacenter and Standard) – Server with Desktop Experience

Updated: Microsoft confirmed that Server Core is supported with Azure File Sync as well. Please check the following article to learn more on How To Install Azure File Sync on Windows Server Core.

PowerShell version 5.1 is the minimum required for Azure File Sync. If you are using Windows Server 2012 R2, then you need to ensure to install at least PowerShell 5.1. You can safely skip this check on Windows Server 2016 and Windows Server 2019.

On Azure

At the time of this writing, Azure File Sync is only available in 25 regions. Microsoft will continue to add support for more regions in the near future. You can check region availability here.

  • Australia East
  • Brazil South
  • Australia Southeast
  • Canada Central
  • Canada East
  • Central India
  • Central US
  • East Asia
  • East US
  • East US2
  • Korea Central
  • Korea South
  • Japan East
  • Japan West
  • North Europe
  • North Central US
  • Southeast Asia
  • South Central US
  • South India
  • UK South
  • UK West
  • West Europe
  • West US
  • West Central US
  • West US 2

Last, you need to create a storage account in Azure, and then create a file share on top of that storage account.

Deploy Azure File Sync

In the following section, I will describe the required steps to automate and deploy Azure File Sync with PowerShell. The same steps can be accomplished through the UI and the Azure Portal.

Step 1 – Prepare Windows Server on-premises to use with Azure File Sync

Open an elevated PowerShell console and run the following command to disable the Internet Explorer Enhanced Security Configuration for Administrators and Users. This is required only for initial server registration in Step 5.

# Disable Internet Explorer Enhanced Security Configuration for Administrators
Set-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Active Setup\Installed Components\{A509B1A7-37EF-4b3f-8CFC-4F3A74704073}" -Name "IsInstalled" -Value 0 -Force

# Disable Internet Explorer Enhanced Security Configuration for Users
Set-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Active Setup\Installed Components\{A509B1A8-37EF-4b3f-8CFC-4F3A74704073}" -Name "IsInstalled" -Value 0 -Force

You can re-enable it by updating the Value to 1 after the server has been registered with Azure File Sync.

Step 2 – Download Azure File Sync Agent

The next step is to download Azure File Sync agent. Open an elevated PowerShell console on Windows server and run the following command. This will download the appropriate Azure File Sync agent based on your Operating System version (Windows Server 2012 R2, Windows Server 2016, and Windows Server 2019).

# Gather the OS version
$osver = [System.Environment]::OSVersion.Version

# Download the appropriate version of the Azure File Sync agent for your OS.
# Azure File Sync Version 6.0
# Windows Server 2019
if ($osver.Equals([System.Version]::new(10, 0, 17763, 0))) {
    Invoke-WebRequest `
        -Uri https://download.microsoft.com/download/1/8/D/18DC8184-E7E2-45EF-823F-F8A36B9FF240/StorageSyncAgent_V6_WS2019.msi `
        -OutFile "StorageSyncAgent.msi" 
}
# Windows Server 2016
elseif ($osver.Equals([System.Version]::new(10, 0, 14393, 0))) {
    Invoke-WebRequest `
        -Uri https://download.microsoft.com/download/1/8/D/18DC8184-E7E2-45EF-823F-F8A36B9FF240/StorageSyncAgent_V6_WS2016.msi `
        -OutFile "StorageSyncAgent.msi" 
}
# Windows Server 2012 R2
elseif ($osver.Equals([System.Version]::new(6, 3, 9600, 0))) {
    Invoke-WebRequest `
        -Uri https://download.microsoft.com/download/1/8/D/18DC8184-E7E2-45EF-823F-F8A36B9FF240/StorageSyncAgent_V6_WS2012R2.msi `
        -OutFile "StorageSyncAgent.msi" 
}
else {
    throw [System.PlatformNotSupportedException]::new("Azure File Sync is only supported on Windows Server 2012 R2, Windows Server 2016, and Windows Server 2019")
}

Note:

In June 2019, Microsoft released Azure File Sync Agent v7.0 in flighting mode which addresses the following issues and improvements: Support for larger file share sizes, Azure Backup file-level restore improvement, Improved cloud tiering recall cmdlet reliability, Support for TLS 1.2 only (TLS 1.0 and 1.1 is disabled), Miscellaneous performance and reliability improvements. Note: This release is being flighted in stages and may not be offered to your server even if it’s configured to automatically update when a new version becomes available. If your server is not configure to receive automatic update (new feature introduced in v6.0) or is not offered to you, then you can upgrade an existing server manually by following the steps documented here KB4490495.

In June 2019, Microsoft released Azure File Sync Agent v6.2 which addresses the following issues: After creating a server endpoint, High CPU usage may occur when background recall is downloading files to the server. Sync and cloud tiering operations may fail with error ECS_E_SERVER_CREDENTIAL_NEEDED due to token expiration. Recalling a file may fail if the URL to download the file contains reserved characters. This update is available for Windows Server 2012 R2, Windows Server 2016 and Windows Server 2019 installations that have Azure File Sync agent version 4.0.1.0 or later installed. Please make sure to install Azure File Sync Agent v6.2, the installation and instructions are documented here KB4489738.

In June 2019, Microsoft announced that Azure File Sync is generally available for Azure Government cloud! Customers in Government Cloud now no longer need special URL hide-key to access their Storage Sync Services. Azure File Sync in Government Cloud can be used with the same v6 agent that a customer would use in public cloud. It is at feature parity with what’s available publicly with Azure Commercial.

In May 2019, Microsoft released Azure File Sync Agent v6.1 which addresses the following issues: Windows Admin Center version 1904 fails to display the agent version and server endpoint configuration on servers which have Azure File Sync agent version 6.0 installed. This update is available for Windows Server 2012 R2, Windows Server 2016 and Windows Server 2019 installations that have Azure File Sync agent version 4.0.1.0 or later installed. Please make sure to install Azure File Sync Agent v6.1, the installation and instructions are documented here KB4489737.

In April 2019, Microsoft also released Azure File Sync Agent v6 which addresses the following improvements and issues that are fixed: Agent auto-update support, Support for Azure file share ACLs, Parallel upload and download sync sessions for a server endpoint, New Cloud Tiering cmdlets to get volume and tiering status, Support for FIPS mode, Miscellaneous reliability improvements for cloud tiering and sync. This update is available for Windows Server 2012 R2, Windows Server 2016 and Windows Server 2019 installations that have Azure File Sync agent version 4.0.1.0 or later installed. Please make sure to install Azure File Sync Agent v6, the installation and instructions are documented here KB4489736.

In April 2019, Microsoft released Azure File Sync Agent v5.2 which addresses the following issues: Reliability improvements for offline data transfer and data transfer resume features, and Sync telemetry improvements. This update is available for Windows Server 2012 R2, Windows Server 2016 and Windows Server 2019 installations that have Azure File Sync agent version 4.0.1.0 or later installed. Please make sure to install Azure File Sync Agent v5.2, the installation and instructions are documented here KB4481061.

In March 2019, Microsoft released Azure File Sync Agent v5.1 which addresses the following issues: Files may fail to sync with error 0x80c8031d, If a sync session or file receives an error 0x80072f78, Files may fail to sync with error 0x80c80203, High memory usage may occur when recalling files, Cloud tiering telemetry improvements. Please make sure to install Azure File Sync Agent v5.1, the installation and instructions are documented here KB4481060.

In February 2019, Microsoft released Azure File Sync Agent v5.0 which includes several improvements and bug fixes: Support for Azure Government cloud, Support for Data Deduplication, Support for offline data transfer, Improved sync performance, Improved monitoring in the portal, Improved scalability and reliability. lease make sure to install Azure File Sync Agent v4.2, the installation and instructions are documented here KB4459989.

In January 2019, Microsoft released Azure File Sync Agent v4.3 which addresses the following issues: Files are not tiered after upgrading the Azure File Sync agent to version 4.x. AfsUpdater.exe is now supported on Windows Server 2019. Miscellaneous reliability improvements for sync. This update is available for Windows Server 2012 R2, Windows Server 2016 and Windows Server 2019 installations that have Azure File Sync agent version 4.0.1.0 or later installed. Please make sure to install Azure File Sync Agent v4.3, the installation and instructions are documented here KB4481059.

Step 3 – Install Azure File Sync agent

The next step is to install the Azure File Sync agent. Open an elevated PowerShell console on your Windows server and run the following command.

# Install the MSI package in quiet mode
Start-Process -FilePath ".\StorageSyncAgent.msi" -ArgumentList "/quiet" -Wait

The Azure File Sync agent consists of three main components:

  • FileSyncSvc.exe: The background Windows service responsible for monitoring changes on Server Endpoints and initiating sync sessions to Azure.
  • StorageSync.sys: The Azure File Sync file system filter, responsible for tiering cold files to Azure Files (when cloud tiering is enabled).
  • PowerShell management cmdlets: PowerShell cmdlets for interacting with the Microsoft.StorageSync Azure Resource Provider. The cmdlets can be found at the following locations (by default):
    • %ProgramFiles%\Azure\StorageSyncAgent\StorageSync.Management.PowerShell.Cmdlets.dll
    • %ProgramFiles%\Azure\StorageSyncAgent\StorageSync.Management.ServerCmdlets.dll

Step 4 – Deploy the Storage Sync Service in Azure

The next step is to create the Storage Sync Service in Azure, but before doing that you need to create the Azure File Sync context.

Open an elevated PowerShell console on your Windows server and run the following script.

Try {
    # Login to Azure
    $AzureSubName = Read-Host "`n Enter Azure Subscription name"
    $AzAcctInfo = Login-AzureRmAccount -Subscription $AzureSubName
    Write-Output "Connecting to Azure Cloud..."
   }
   Catch {
     Write-Output "Cannot connect to Azure environment. Please check your credentials and Azure Subscription name. Exiting!"
     Break
   }

# The default location of the Azure File Sync Agent
$AFSAgentPath = "C:\Program Files\Azure\StorageSyncAgent"

# Import the Azure File Sync PowerShell management cmdlets
Import-Module "$AFSAgentPath\StorageSync.Management.PowerShell.Cmdlets.dll"

# Get the Azure subscription ID and store it in a variable
$AzureSubID = $AzAcctInfo.Context.Subscription.Id

# Get the Azure Active Directory tenant ID and store it in a variable
$AzTenantID = $AzAcctInfo.Context.Tenant.Id

# Enter any of the supported Azure region you want to deploy Azure File Sync into
# https://docs.microsoft.com/en-us/azure/storage/files/storage-sync-files-planning#region-availability
$AzRegion = Read-Host "`n Enter Azure Region name"

# Enter the resource group name to deploy the Storage Sync Service into
$AzResourceGroup = Read-Host "`n Enter Azure Resource Group name"
New-AzureRmResourceGroup -Name $AzResourceGroup -Location $AzRegion

# The following command creates an AFS context
Login-AzureRmStorageSync `
    –SubscriptionId $AzureSubID `
    -ResourceGroupName $AzResourceGroup `
    -TenantId $AzTenantID `
    -Location $AzRegion

# Create Azure Storage Sync Service
$AFSStorageSyncName = Read-Host "`n Enter the desired name of your Storage Sync Service"
New-AzureRmStorageSyncService -StorageSyncServiceName $AFSStorageSyncName

Do not close the PowerShell window after running the script above, this will help to run subsequent Azure File Sync PowerShell commands in the next steps with minimal repetition of parameters and authentication.

Step 5 – Register Windows Server with Storage Sync Service

In this step, you need to register your server on-premises with Azure Storage Sync Service. Open an elevated PowerShell console on your Windows server and run the following command.

$RegisteredServer = Register-AzureRmStorageSyncServer -StorageSyncServiceName $AFSStorageSyncName

Registering your Windows Server with a Storage Sync Service establishes a trust relationship between your server on-premises and the Storage Sync Service in Azure. If you are deploying Azure File Sync on a node which is member of a Failover Cluster, then you need to repeat the same steps described above on each server node in a Failover Cluster.

Step 6 -Create a Storage account and file share

As mentioned earlier in this article, we need to create a storage account and file share in Azure before we start syncing. Open an elevated PowerShell console and run the following script.

# Create a storage account with desired name
$AzStorageAccountName = Read-Host "`n Enter the desired storage account name in lowercase letters only"

$AzStorageAccount = New-AzureRmStorageAccount `
        -Name $AzStorageAccountName `
        -ResourceGroupName $AzresourceGroup `
        -Location $Azregion `
        -SkuName Standard_LRS `
        -Kind StorageV2 `
        -EnableHttpsTrafficOnly:$true


# Create an Azure file share within the desired storage account
$FileShareName = Read-Host "`n Enter the desired Azure file share name in lowercase letters only"
$FileShare = New-AzureStorageShare -Context $AzStorageAccount.Context -Name $fileShareName

Step 7- Create a Sync Group

In this step, we need to create a sync group. A sync group is the basic topology unit of Azure File Sync, it defines the relationship of a sync partnership.

Open an elevated PowerShell console and run the following command.

$SyncGroupName = Read-Host "`n Enter the desired Sync Group Name"
New-AzureRmStorageSyncGroup -SyncGroupName $SyncGroupName -StorageSyncService $AFSStorageSyncName

Step 8 – Create a cloud endpoint

In this step, we need to create a cloud endpoint. A cloud endpoint is a pointer to an Azure file share. All server endpoints (Step 9) will sync with a cloud endpoint, making the cloud endpoint the hub.

Open an elevated PowerShell console and run the following command.

# Create a cloud endpoint
New-AzureRmStorageSyncCloudEndpoint `
    -StorageSyncServiceName $AFSStorageSyncName `
    -SyncGroupName $SyncGroupName `
    -StorageAccountResourceId $AzStorageAccount.Id `
    -StorageAccountShareName $fileShare.Name

Step 9 – Create a server endpoint

In the last step, we need to create a server endpoint. A server endpoint represents a path on a registered server on disk on-premises, such as a folder on a server volume. The path must be on a non-system volume if you plan to enable Cloud Tiering. Cloud Tiering cannot be enabled on the system volume.

Open an elevated PowerShell console on the registered server and run the following script.

$ServerEndpointPath = Read-Host "`n Enter the desired path on your registered server which is not on the system volume"

$DriveLetter = $ServerEndpointPath.Substring(0,1)
$volume = Get-Volume -DriveLetter $DriveLetter
$volumeFreeSpace = “{0:P0}” –F ($volume.SizeRemaining/$volume.Size)
$volumeFreeSpace = $volumeFreeSpace.Substring(0,3)
$volumeFreeSpacePercentage=[int]$volumeFreeSpace

# Create server endpoint
New-AzureRmStorageSyncServerEndpoint `
    -StorageSyncServiceName $AFSStorageSyncName `
    -SyncGroupName $syncGroupName `
    -ServerId $registeredServer.Id `
    -ServerLocalPath $serverEndpointPath `
    -CloudTiering $true `
    -VolumeFreeSpacePercent $volumeFreeSpacePercentage

Testing Azure File Sync

Now you have Azure File Sync deployed and ready to use. You can copy some files to the server endpoint under the path specified in Step 9. Based on your Internet bandwidth and the size of the files, storage sync will automatically kick off behind the scene and the files will be synced to Azure File share as shown in the following screenshot.

Since I enabled cloud tiering and I set the volume of free space percentage to maximum 100%.

All the files have been tiered to Azure file share. The following screenshot shows the actual size of the file and the size locally on disk.

 

And there you have it.

Summary

Azure File Sync extends on-premises files servers into Azure providing cloud benefits while maintaining performance and compatibility.

Azure File Sync provides:

  • Multi-site access – provide write access to the same data across Windows servers and Azure Files.
  • Cloud tiering – store only recently accessed data on local servers.
  • Integrates with Azure backup – no need to back up your data on-premises.
  • Fast disaster recovery – restore file metadata immediately and recall data as needed.

I hope you find this article useful.

__
Thank you for reading my blog.

If you have any questions or feedback, please leave a comment.

-Charbel Nemnom-

Advertisements
About Charbel Nemnom 459 Articles
Charbel Nemnom is a Cloud Architect and Microsoft Most Valuable Professional (MVP), totally fan of the latest's IT platform solutions, accomplished hands-on technical professional with over 17 years of broad IT Infrastructure experience serving on and guiding technical teams to optimize performance of mission-critical enterprise systems. Excellent communicator adept at identifying business needs and bridging the gap between functional groups and technology to foster targeted and innovative IT project development. Well respected by peers through demonstrating passion for technology and performance improvement. Extensive practical knowledge of complex systems builds, network design and virtualization.