In this article, we will show you how to protect and copy Azure file shares snapshots from one storage account to another and overcome the Geo DR challenges.
Table of Contents
Introduction
Azure Files offers shared storage for applications using the standard SMB 3.0 protocol. Microsoft Azure virtual machines and cloud services can share file data across application components via mounted shares, and on-premises applications can access file data in a share via the File storage API.
Applications running on Azure virtual machines can mount a File storage share to access file data, just as a desktop application would mount a typical SMB share. Any number of Azure virtual machines or roles can mount and access the File Storage share simultaneously.
Microsoft also introduced the Azure File Sync service which allows you to centralize your file shares in Azure Files, whilst maintaining the compatibility of an on-premises file server with all the flexibility and performance benefits provide. For more information about Azure File Sync and how to get started, please check the following step-by-step guide.
On May 27, 2020, the Azure Backup team in coordination with the Azure Files team announced the general availability of the long-awaited feature “Soft Delete” to protect your file share(s) from accidental deletion and malicious actors. However, the snapshots for Azure Files are stored within the same Azure Storage account. In other words, the snapshots are NOT transferred to the Recovery Services Vaults as we have with Azure IaaS/SQL VM backup today.
What if your organization’s policy dedicates to having a disaster recovery for the exiting Azure file shares to be stored in another region?
Backup Overview of Azure File Shares
At the time of this writing, when you configure Azure Backup for Azure File Share, what will happen is the following:
1) You create or select an existing Recovery Services Vaults.
2) You enable backup and select Azure File Share.
3) Then you select the source Azure storage account where the Azure File share(s) reside. Then the storage account will be registered with the Recovery Services Vaults. No backup is transferred from the storage account to the Recovery Services Vaults which is known as hardening.
4) The next step is to select one or more file share(s) which you want to protect.
5) Azure Backup enables “Soft Delete” on the storage account with the default retention period of 14 days.
6) Last but not least, you can choose or create a new backup policy (daily, weekly, monthly, or yearly backup up to 10 years only through the Azure Portal).
7) Finally, when each backup job runs, the Azure Backup service will ensure that the soft delete is always turned on.
At the moment you can take storage snapshots of the Azure file share but they remain bound to the same storage account, which is quite risky from a disaster recovery point of view.
In this article, we will show you how to protect and copy Azure file share snapshots from one storage account to another by leveraging AzCopy sync to overcome the Geo DR challenges.
Prerequisites
The prerequisites are very simple as follows:
1) Download AzCopy V10.5.1 from here, or jump into the Azure Cloud Shell session, AzCopy is included as part of the Cloud Shell. Please make sure you are using the latest AzCopy version otherwise the process will fail.
2) You need to have two different storage accounts either in the same region, same subscription or in different regions and subscriptions. From a disaster recovery perspective, it’s more likely to have the second storage account in a different region.
3) You also need to have one or more Azure file shares created in each storage account.
4) Download Microsoft Azure Storage Explorer from here if you don’t have it yet, I will use it to create the Shared Access Signature (SAS) tokens. You can also generate SAS tokens using the Azure Portal, as well as using PowerShell.
As a side note, SAS is more secure than the storage account keys. One of the main reasons is with SAS, you can ACL the IPs that can access the account, you can control the permissions on the account in a more granular fashion, and when the token will expire, as well as which service you want to have access to (Blobs, Files, Queues, Tables). Please note that SAS tokens are signed with your key, so regenerating your storage account key should invalidate the SAS token as well, keep that in mind!
Sync Snapshots Between Azure File Shares
Assuming you have some files in your file share. Once you are ready, take the following steps:
The first step is to create a snapshot (+ Add snapshot) on the source file share. In this example, I will be using the manual approach. Please note that all steps can be automated as well.
In this example, I have the following snapshots created on the first Azure File Share named (afs-atp-asc). I will copy the following snapshot name: 2020-07-31T14:55:42.0000000Z
In the second storage account, I have a file share named (geo-dr-afs), and I don’t have any files yet as shown in the below figure.
The next step is to sync a specific (snapshot) from the source storage account/file share to another storage account/file share with the SAS key (the key has been intentionally changed in this example). The full syntax is as follows:
azcopy sync "https://sourcestorageacc.file.core.windows.net/afs-atp-asc?sv=2019-12-12&xxxx=xxxxxxxxxB0RmdZuw6jXE%3D&sharesnapshot=2020-07-31T14:55:42.0000000Z" "https://targetstorageacc.file.core.windows.net/geo-dr-afs/?sv=2019-12-12&ss=xxxxxxxse=2021-07-31T23:07:39Z&xxxx-07-1&sig=xxxxx3D"This command will copy the snapshot including all the files from the source file share and sync the contents to the destination Azure file share in the second storage account.
The output will show the total number of files at the Source, Destination, and the Total Number of Copy Transfers Completed which are 49 files, as well as the time in Minutes (1.6) for ~ 5GiB (the SAS key has been intentionally obscured in this demo).
Open the second file share in the second storage account and verify that all the files are synced over.
The next step is to take a snapshot of the second file share, this will result in consistent snapshots (identical) at the source and target file shares.
Finally, you need to repeat the same process described above to sync and replicate more snapshots.
Summary
By using AzCopy sync, you can do differential snapshot copies to another storage account (file share). The workflow is as follows:
- Take a snapshot of the source file share.
- Copy the snapshot to the target file share using AzCopy sync.
- Take a snapshot of the target file share. The result is both snapshots are the same.
- Finally, repeat steps (1->3) to replicate more snapshots.
Please note that synchronization is one-way. In other words, you choose which of the two endpoints is the source and which one is the destination.
By using this approach, you get Geo DR file share in another Azure region, as well as maintaining the same recovery time objective (RTO) as the primary file share with snapshot consistency.
To prevent accidental deletions on the source file share, please make sure to enable the new soft delete feature on your Azure file shares as described here.
AzCopy Sync will only transfer new files and the files that changed, and can also remove deleted files from the target file share. For more information about AzCopy Sync syntax with additional parameters, please check the following document from Microsoft.
How to automate…
As described in this article, this is a manual approach and not so efficient to scale. As of today, if you want to have a quick DR of your Azure File Shares, then AzCopy sync is the way to go.
Stay tuned for the next article where I will describe how to automate the entire process. Check the next article here where we described how to automate the backup of Azure File Shares for Disaster Recovery.
That’s it there you have it.
__
Thank you for reading my blog.
If you have any questions or feedback, please leave a comment.
-Charbel Nemnom-
Great blog. It clarified some options for me :)
I will be working on the next step too then: automating this.
Thank you for your article, I have just tested and just to notify you that your command is not working as expected for me.
So I changed the part : xxx?snapshot=2020-07-31T14:55:42.0000000Z;svxxxx by xxx?svxxx&sharesnapshot=2020-07-31T14:55:42.0000000Z
Which is working better and synchronize only expected files from my snapshot because other URI synchronize all files, tested by added a file after making a snapshot.
Thank you Jon for testing and confirming the syntax. Please note that Microsoft has changed the share snapshot architecture since this article was written. I have updated it to reflect the new syntax. Please check this guide about automating the backup of file share snapshots and let me know your feedback. Thank you again!