Table of Contents
Azure File Sync can help you centralize your files in Azure and then install a sync agent on Windows Server whether it’s on-premises or in Azure (IaaS VM) to provide fast local access to your files. Your server and Azure Files are constantly in sync, so you have one centralized location for your files with multi-site access powered by fast local cache and cloud tiering.
The cloud tiering feature enables frequently accessed files to be cached locally such that the full file content is present on the server, whereas less frequently accessed files are tiered to the cloud. The tiered files (AFS reparse points) will be recalled on-demand when a user or application accesses it on the local server.
For more information about Azure File Sync, please check my previous articles here.
Starting with Azure File Sync version 4.0 and above, Microsoft added a new secure Windows attribute to all tiered files known as (FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS).
This new attribute will improve Antivirus (AV) and Backup products that would otherwise wastefully download cold files from the cloud. Because antivirus works by scanning files for known malicious code, an antivirus/backup product might cause the recall of tiered files which will incur additional costs. So it’s recommended to consult with your software vendor to learn how to configure their solution to skip reading files with this attribute set (many do it automatically). Microsoft’s in-house antivirus solutions, like Windows Defender and System Center Endpoint Protection (SCEP), both automatically skip reading files that have this attribute set.
The good news is, Microsoft just released a new lightweight test kit that helps identify potential interoperability issues between Azure File Sync and antivirus solutions. This test kit primarily helps identify the tiered file reads caused by AV that likely can be avoided to prevent unexpected data egress from the cloud. The decision to scan/read the tiered file or not ultimately belongs to the AV software.
In this blog post, I will show you how to install the new Azure File Sync (AFS) Antivirus Compatibility Suite, so you can test all operations of Azure File Sync and validates the behavior of Antivirus using its comprehensive set of tests, and then produces user-friendly test results.
AFS Antivirus Compatibility Test Suite
The Azure File Sync (AFS) Antivirus Compatibility Test Suite is developed as a stand-alone product. It doesn’t need an Azure File Share or Azure subscription with storage sync setup to run the tests. Better yet, it doesn’t even need the agent installed. All file system interactions critical to AFS are implemented by the tests.
The AFS AV test suite’s filter listener mocks the FileSyncSvc.exe implementation. In the case of an antivirus product inappropriately recalling tiered files – for example, during a regularly scheduled scan – the filter listener will detect that the incoming read request was not ‘expected’ and so the test will fail. Based on the test results, antivirus vendors can make appropriate changes to the behavior of their product with Azure File Sync.
Setup AFS Antivirus Test Suite
The Azure File Sync (AFS) Antivirus Compatibility Test Suite can be downloaded from the following location.
Once the setup .zip file is downloaded, you can extract it to a location on your File Server such as D:\AFS-AV-Test
Please note that the AV test suite can be installed on Windows Server 2012 R2, Windows Server 2016, or Windows Server 2019.
Next, download and install the Windows Driver Kit (WDK) from the following location.
Once WDK is installed, the Test Authoring and Execution Framework (TAEF) can be found at the WDK installation path under C:\Program Files (x86)\Windows Kits\10\Testing\Runtimes\TAEF. Note the TE.exe command, we will use it in the next section.
Go back to the extracted location and install the StorageSync.sys filter, you need to right-click on StorageSync.inf file and select Install.
To verify that the filter is correctly installed and loaded, you need to launch Windows command prompt and run the following command: fltmc. If you don’t see the StorageSync filter, then try to load it with this command: fltmc load StorageSync
Now you need to set up the AV background scan, open the runbackgroundscan.cmd in the same folder and modify this file to execute the command needed to run a background scan. In this example, I am using Windows Defender as an antivirus, the %1 is the path to scan that we will specify when we run the test suite in the next section. If your AV product does not support initiating a background scan in this way, then you will have to complete this test manually.
Test AFS Antivirus Execution
Open command prompt (cmd.exe) in elevated mode and switch to C:\Program Files (x86)\Windows Kits\10\Testing\Runtimes\TAEF and then run the following command:
te.exe "D:\AFS-AV-Test\AVCompatTestSuite.dll" /p:TargetDirectory=D:\Backup\ /p:OperationDelaySeconds=10 > output.txt
Please note that the Target Directory should be on a non-system (C:) volume. OperationDelaySeconds is the sleep time between critical test operations such that it allows time for the AV to do its job. The delay is introduced after files creation, after tiering the file and after performing test actions on files and before the test validation.
The output.txt file test result is saved under the same folder: C:\Program Files (x86)\Windows Kits\10\Testing\Runtimes\TAEF
This file contains all the tests that were performed by the test suite. For granular test execution, please check the Readme.pdf file that comes with the AV test suite.
If you open the output.txt file, you will see a list of all tests, as well as a summary of the AVCompatTest. Please note that the terms “tiered file” and “ghosted file” are synonymous, hence Microsoft used of the term “ghosted’ in the test names. As you can see in the following example, none of the test Failed=0, thus, Windows Defender AV is fully compatible with Azure File Sync.
For the detailed description of each test that runs as part of this suite, along with what to do if it fails. Please check the Readme.pdf document that comes with the AV test suite.
How it works…
The Azure File Sync Antivirus Compatibility Test Suite has two components:
- StorageSync.sys: The Azure File Sync file system filter, which is responsible for tiering files to Azure Files (when cloud tiering is enabled).
- AVCompatTestSuite.dll: Defines a comprehensive list of tests, that are executed using the TAEF framework and produces easy to understand results.
The test suite utilizes the Test Authoring and Execution Framework (TAEF) that is publicly available as part of the Windows Driver Kit (WDK) or a standalone installation.
The AV test validates that a background scan behaves appropriately with placeholders. It is intended to simulate a scheduled AV scan of the user’s file system. Specifically, the following rules must be obeyed in a background scan:
- Dehydrated placeholders are not scanned, and therefore not hydrated.
- Hydrated files are scanned. This is validated by tracking the file’s access time for the duration of the scan.
This test requires a modification to runbackgroundscan.cmd to work for the AV product being tested. Please make sure to update it according to your AV product. This test will fail if any of the above conditions are not met.
Azure File Sync extends on-premises file servers into Azure providing cloud benefits while maintaining performance and compatibility. Azure File Sync provides:
- Multi-site access – provide write access to the same data across Windows servers and Azure Files.
- Cloud tiering – store only recently accessed data on local servers.
- Integrates with Azure backup – no need to back up your data on-premises.
- Fast disaster recovery – restore file metadata immediately and recall data as needed.
I hope you find this guide useful. To learn more about Azure File Sync, please check the following guides.
Thank you for reading my blog.
If you have any questions or feedback, please leave a comment.