Experiencing Slow File Recall With Azure File Sync When Cloud Tiering Is Enabled

4 min read

Introduction

In July 2018, Microsoft announced the GA release for Azure File Sync. With Azure File Sync, you can centralize your files in Azure and then install a sync agent on Windows Server whether it’s on-premises or in Azure (IaaS VM) to provide fast local access to your files. Your server and Azure Files are constantly in sync, so you have one centralized location for your files with multi-site access powered by fast local cache and cloud tiering.

Cloud tiering is an optional feature of Azure File Sync in which frequently accessed files are cached locally on the server while all other files are tiered to Azure Files based on policy settings. When a file is tiered, the Azure File Sync file system filter (StorageSync.sys) replaces the file locally with a pointer, or reparse point. The reparse point represents a URL to the file in Azure Files.

When a user opens a tiered file, Azure File Sync seamlessly recalls the file data from Azure Files without the user needing to know that the file is actually stored in Azure. For more information about the cloud tiering option, please check the official documentation from Microsoft.

A while ago, I blogged about how to control Azure File Sync bandwidth with network QoS. Please have a quick read before you continue with the remainder of this article.

The issue

I have configured a new server endpoint and I am using network throttling QoS as described in this article.

For this particular server endpoint, I was using the Invoke-StorageSyncFileRecall cmdlet to pull down a large amount of data. I noticed a slow file download and I start wondering if this experience is related to the network throttling option.

In this particular example, to recall 9 GiB of data took around 13 minutes as shown in below output.

The network and CPU performance was about 153 Mbps and 15% CPU time.

As confirmed by Microsoft, when you throttle the network, Azure File Sync does not apply (throttle) when a tiered file is accessed or the Invoke-StorageSyncFileRecall cmdlet is used (download). So if Cloud Tiering is enabled, the majority of the traffic that is throttled is Upload. If a user accesses a tiered file or the Invoke-StorageSyncFileRecall cmdlet is used, the request is NOT throttled. Throttling applies on download if the server is downloading a file change that was made on another server endpoint and the file was not accessed by a user or application.

So network throttling is not the issue when you recall or access a tiered file.

The solution

After a bit of investigation, I found out that when you use the Invoke-StorageSyncFileRecall PowerShell cmdlet, it has an additional parameter called (-ThreadCount). The thread count determines how many files can be recalled in parallel.

Now when you run the Invoke-StorageSyncFileRecall cmdlet without the -ThreadCount parameter. The default is 4 files are recalled in parallel.

In this example, I increased the -ThreadCount to 16 and the recall time dropped to 8 minutes instead of 13 minutes before. This is for a small amount of data (9 GiB). Imagine that you have 10+ TB of data that you want to recall.

If we look at the performance now, we see 179 Mbps and 20% CPU time compare to 4 threads before. This is expected as we are recalling 16 files in parallel.

How many files can you recall in parallel? The default is 4, and it can be increased up to 32.

There is also an additional parameter you may consider using is (-Order CloudTieringPolicy). Specifying -Order CloudTieringPolicy will recall the most recently modified files first.

Invoke-StorageSyncFileRecall -Path D:\Data -Order CloudTieringPolicy -ThreadCount 16 -Verbose

Hope this helps!

Summary

As described in this article, you can easily increase the thread count by running a simple PowerShell command on the server endpoint.
The thread count determines how many files can be recalled in parallel. By default Azure File Sync agent is set to recall 4, however, you can increase it up to 32 threads.

Azure File Sync extends on-premises file servers into Azure by providing cloud benefits while maintaining performance and compatibility. Azure File Sync provides:

  • Multi-site access – provide write access to the same data across Windows servers and Azure Files.
  • Cloud tiering – store only recently accessed data on the local server(s) and save on capacity storage.
  • Integrates with Azure backup – no need to back up your data on-premises.
  • Fast disaster recovery – restore file metadata immediately and recall data as needed.

I hope you find this guide useful. To learn more about Azure File Sync, please check the following guides.

__
Thank you for reading my blog.

If you have any questions or feedback, please leave a comment.

-Charbel Nemnom-

About Charbel Nemnom 525 Articles
Charbel Nemnom is a Cloud Architect, ICT Security Expert and Microsoft Most Valuable Professional (MVP), totally fan of the latest's IT platform solutions, accomplished hands-on technical professional with over 17 years of broad IT Infrastructure experience serving on and guiding technical teams to optimize the performance of mission-critical enterprise systems. Excellent communicator is adept at identifying business needs and bridging the gap between functional groups and technology to foster targeted and innovative IT project development. Well respected by peers through demonstrating passion for technology and performance improvement. Extensive practical knowledge of complex systems builds, network design, business continuity, and cloud security.

1 Comment

  1. Hello Mr. Nemnom,

    I am currently trying to fix an issue we have with the Azure File Sync Date Policy and so far every remediation step I tried has failed. As someone who has commited that much time into AFS, perhaps you know how to resolve our problem:

    We have an Azure File Sync setup running on a W2k12 R2 fileserver. We initialized Cloud Tiering with the VolumeFreeSpace Policy to 20% free space which worked without any issues. After that we set the Date Policy to tier files older than 550 days. Before we set up AFS we ran an analysis where we found out that there is around 1TB of data not being accessed for more than 550 days, so we expected the AFS to correctly tier those files and free up the space. However it did not work.

    I tried many things, including restarting the service, the server, deactivating and activating the Date Policy many times and the last thing I did was doing an exclusion on SCEP for the FileSyncSvc.exe.
    The result after restarting the Service then was a complete Heatstore Maintenance, which sadly didnt result in the Date Policy to kick in.

    We have the latest Agent (9.1) installed. On a sidenote, we are currently having issues with a constant upload-stream because of .pst files causing sync errors and trying to sync almost 24/7

    I would appreciate any help if possible! :)

    Best regards,

    Mark Brigaldino

This site uses Akismet to reduce spam. Learn how your comment data is processed.