How to Get Azure File Sync Cloud Tiering Efficiency With PowerShell

8 min read

In this article, I will share with you how to get Azure File Sync Cloud tiering efficiency for a server endpoint with PowerShell.

Updated – 07/11/2020 – Azure File Sync Agent v11.1 is released. If you are using version 11.1 or later, please check the following section to get Azure File Sync cache efficiency.

Introduction

Azure Files offers shared storage for applications using the standard SMB 3.0 protocol. Microsoft Azure virtual machines and cloud services can share file data across application components via mounted shares, and on-premises applications can access file data in a share via the File storage API.

Applications running on Azure virtual machines can mount a File storage share to access file data, just as a desktop application would mount a typical SMB share. Any number of Azure virtual machines or roles can mount and access the File Storage share simultaneously.

Microsoft also introduced the Azure File Sync service which allows you to centralize your file shares in Azure Files, whilst maintaining the compatibility of an on-premises file server with all the flexibility and performance benefits that provide.

One of my blog readers has asked me about the possibility to read and get the Cloud Tiering efficiency from the Azure Portal using PowerShell.

How to Get Azure File Sync Cloud Tiering Efficiency With PowerShell 1

In this article, I will share with you how to read and get the Azure File Sync cloud tiering efficiency with PowerShell.

Prerequisites

To follow this article, you need to have the following:

  1. Azure subscription. If you don’t have an Azure subscription, you can create a free one here.
  2. Azure storage account – To create a general-purpose storage account, you can follow the instructions described here.
  3. Step-by-step guide on how to get started with Azure File Sync.

Please note that this guide is based on Azure File Sync agent v10.1. Things might change in the next update.

Cloud Tiering Efficiency

At the time of this writing, to get the cloud tiering efficiency for a server endpoint, this has to be done locally on the server by looking at the events in Event Viewer and not on Azure.

To illustrate the entire process, I will use the following Tiering Efficiency as my example.

How to Get Azure File Sync Cloud Tiering Efficiency With PowerShell 2

Assuming you have all the prerequisites in place, log in to the server endpoint and take the following steps:

Is Cloud Tiering enabled?

First, we need to check if cloud tiering is enabled for a specific server endpoint (share path), you can run the following PowerShell cmdlet to verify the share has tiering enabled. I will put the output in a variable called $tiering.

#Check if tiering is enabled
$tiering = Get-WinEvent -FilterHashtable @{LogName='Microsoft-FileSync-Agent/Telemetry';Id='9001'} | Where-Object Message -match "IsTieringEnabled:true"

How to Get Azure File Sync Cloud Tiering Efficiency With PowerShell 3

Tiering Efficiency

Once we get the tiering status for a specific share, then we can query the tiering efficiency.

The cloud tiering efficiency is divided into 3 sections as follows:

  • The efficiency for each server endpoint.
  • The efficiency per volume.
  • The cache rate.

You need to pull out each section from Event Viewer and parse the output by following the steps below:

Tiering Efficiency per server endpoint

To get the tiering efficiency per share (server endpoint), you need to run the following PowerShell cmdlet. I will put the output in a variable called $efficiencyEndpoint.

#Get the efficiency per server endpoint (share)
$efficiencyEndpoint = Get-WinEvent -FilterHashtable @{LogName='Microsoft-FileSync-Agent/Telemetry';Id='9031'} | Where-Object Message -Match $tiering[0].Message.Substring(46,36)

Then, I will do some PowerShell parsing to get the needed information to calculate the tiering efficiency per share.

$efficiencyEndpoint = $efficiencyEndpoint[0].Message -split "`n"
$efficiencyEndpoint | Select-String -Pattern 'Number of'

Next, you need to get the [Number of tiered files in share (Logical size GB)], then the [Number of tiered files in share (Physical size GB)], and the [Total number of files processed (Logical size GB)] as shown in the output below:

How to Get Azure File Sync Cloud Tiering Efficiency With PowerShell 4

The formula to calculate the efficiency for server endpoint is [Number of tiered files in share (Logical size GB)] – [Number of tiered files in share (Physical size GB)] / [Total number of files processed (Logical size GB)] X 100.

So in my example, this looks like this (3.942 – 0) / 3.962 * 100 = 99.49%. We can verify that by looking at the Cloud Tiering Efficiency (Space savings) in the Azure Portal.

How to Get Azure File Sync Cloud Tiering Efficiency With PowerShell 5

Tiering Efficiency per volume

Next, is to get the tiering efficiency per volume. You can run the following PowerShell cmdlet to get the efficiency per volume based on the 9004 events.

#Get the efficiency per volume
$efficiencyVolume = Get-WinEvent -FilterHashtable @{LogName='Microsoft-FileSync-Agent/Telemetry';Id='9004'}
$efficiencyVolume = $efficiencyVolume[0].Message -split "`n"
$efficiencyVolume | Select-String -Pattern 'Volume Size'
$efficiencyVolume | Select-String -Pattern 'Volume Free Space %'

In my example, the output looks like this:

How to Get Azure File Sync Cloud Tiering Efficiency With PowerShell 6

We can verify that by looking at the Cloud Tiering Efficiency (Volume size, Effective volume policy, and the Current volume free space) in the Azure Portal.

How to Get Azure File Sync Cloud Tiering Efficiency With PowerShell 7

Cache Hit Rate

Finally, is the Cache hit rate per server endpoint (share). You can run the following PowerShell cmdlet to get the Cache hit rate based on the 9071 events. Please note that for the Cache hit rate, the 9071 events fired almost every hour (if the system is busy and accesses is happening). I will put the output in a variable called $cacheRate.

#Get Cache hit rate 
$cacheRate = Get-WinEvent -FilterHashtable @{LogName='Microsoft-FileSync-Agent/Telemetry';Id='9071'}
$cacheRate[0] | ForEach {$_.Message -split ',' | Where-Object{$_ -match 'Total'} }

In my example, the output looks like this. Take a note for the TotalCacheHitBytes, TotalCacheMissBytes, and TotalBytesRecalledFromNetwork.

How to Get Azure File Sync Cloud Tiering Efficiency With PowerShell 8

The formula to calculate the Cache hit rate for server endpoint is: ((100 – (TotalCacheHitBytes / (TotalCacheHitBytes + TotalBytesRecalledFromNetwork) X 100)).

So in my example, this looks like this (100 – (9175040 / (9175040 + 20971520) X 100)) = 70%.

We can verify that by looking at the Tiering Efficiency (Cache hit rate) in the Azure Portal.

How to Get Azure File Sync Cloud Tiering Efficiency With PowerShell 9

There’s more…

As noted in this article, to look at this information locally on the server you should be able to view it in the Event Viewer under the Telemetry channel.

Please note that the events get rolled over (delete it) after it reaches 1028 KB. So, if you want to keep the events for a longer time, you might want to adjust the setting appropriate log size by right-clicking on Telemetry channel => Properties as shown in the figure below and change it to a higher size (e.g. 4096 KB).

How to Get Azure File Sync Cloud Tiering Efficiency With PowerShell 10

That’s it there you have it.

Get-StorageSyncFileAccessPattern

Starting with Azure File Sync agent v11.1, Microsoft added a new PowerShell cmdlet that will report the cache efficiency data based on calculations that were done during daily tiering maintenance scans.

The access pattern cmdlet provides the ability to view the access patterns and cache efficiency on the local server over a period of time.

How to use it

After you upgrade the agent to version 11.1, you need to import the Storage Sync PowerShell module and then run the “Get-StorageSyncFileAccessPattern” cmdlet a shown in the example below:

# Import StorageSync PowerShell module
Import-Module "C:\Program Files\Azure\StorageSyncAgent\StorageSync.Management.ServerCmdlets.dll"

# Syntax
Get-StorageSyncFileAccessPattern [-ServerEndpointPath] <string> 
# Example
Get-StorageSyncFileAccessPattern -ServerEndpointPath D:\Data

In my example, the output looks like this.

How to Get Azure File Sync Cloud Tiering Efficiency With PowerShell 11

The AccessDaysBucketStart and AccessDaysBucketEnd provide the days relative to today as follows:

  • 0-7 means [Today-7] days of access pattern.
  • 7-30 means access patterns from 7 days ago going back 30 days.
  • 30-90 means access patterns from 30 days ago going back 90 days.
  • 90-180 means access patterns from 90 days ago going back 180 days.
  • 180-365 means access patterns from 90 days ago going back 1 year, and so on…

If the AccessedFileCount > 0 in any particular bucket, only then the data is of interest to us. As shown in my example below, we see that 17 files were accessed in the range [30-90] days (e.g. within the last 3 months) and the CacheHitPercent was 89%. You might think that the data is not making much sense due to the fact that all 17 files that were accessed are non-tiered files (AccessedNonTieredFileCount).

How to Get Azure File Sync Cloud Tiering Efficiency With PowerShell 12

When the system will see more realistic access patterns over time, then the data will start to make more sense. Please note that as soon as you upgrade to the v11.1 agent that supports this cmdlet, you will not have useful data to interpret it properly in the cmdlet output. You need to look at the data again after a month or so.

At the time of this writing, the “Get-StorageSyncFileAccessPattern” cmdlet is still in the “Beta” release, you can expect more updates in the near future.

Please note that the same data shown from the “Get-StorageSyncFileAccessPattern” cmdlet is also logged under the Event Id 9078.

How to Get Azure File Sync Cloud Tiering Efficiency With PowerShell 13

Summary

As described in this article, you can retrieve and get the Azure File Sync Cloud tiering efficiency using PowerShell locally from the server. I hope that Microsoft will improve and include a convenient way of getting this information via a single PowerShell cmdlet instead of parsing multiple outputs and do the calculation. But until Microsoft update their PowerShell module, you can leverage this approach to check the cloud tiering efficiency.

As noted above, this guide is based on Azure File Sync agent v10.1. Things might change in the next agent update. Check here!

Please note that in most cases, the frequency of the telemetry is happening once a day but in some larger-scale systems it can take many days to complete the scan, and hence data can be many days old. Microsoft recommends looking at the Azure Portal to understand the cache efficiency.

Azure File Sync extends on-premises file servers into Azure by providing cloud benefits while maintaining performance and compatibility. Azure File Sync provides:

  • Multi-site access – provide write access to the same data across Windows servers and Azure Files.
  • Cloud tiering – store only recently accessed data on the local server(s) and save on capacity storage.
  • Integrates with Azure backup – no need to back up your data on-premises.
  • Fast disaster recovery – restore file metadata immediately and recall data as needed.

I hope you find this guide useful. To learn more about Azure File Sync, please check the following step by step guides.

__
Thank you for reading my blog.

If you have any questions or feedback, please leave a comment.

-Charbel Nemnom-

Related Posts

About Charbel Nemnom 579 Articles
Charbel Nemnom is a Cloud Architect, Swiss Certified ICT Security Expert, Microsoft Most Valuable Professional (MVP), and Microsoft Certified Trainer (MCT), totally fan of the latest's IT platform solutions, accomplished hands-on technical professional with over 17 years of broad IT Infrastructure experience serving on and guiding technical teams to optimize the performance of mission-critical enterprise systems. Excellent communicator is adept at identifying business needs and bridging the gap between functional groups and technology to foster targeted and innovative IT project development. Well respected by peers through demonstrating passion for technology and performance improvement. Extensive practical knowledge of complex systems builds, network design, business continuity, and cloud security.

1 Comment

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.