Updated – 08/02/2021 – Soft delete for Azure file shares is now enabled by default!
Updated – 17/12/2020 – Azure file shares backup constraints!
Updated – 30/10/2020 – Soft delete for Azure file shares is now generally available in all regions!
Updated – 07/08/2020 – Azure Backup integration into Azure file share blade!
Updated – 28/05/2020 – Azure Backup now provides protection against accidental deletion of Azure file shares!
Updated – 16/05/2020 – Azure Backup for Azure file share supports a maximum of 11 snapshots per day!
With Azure File Sync, you can centralize your files in Azure and then install a sync agent on Windows Server on-premises or in Azure (IaaS VM) to provide fast local access to your cloud files. Your local server and Azure are constantly syncing, so you have one centralized location for your files with multi-site access powered by fast local caches and cloud tiering. What cloud tiering does, it enables frequently accessed files to be cached locally such that the full file content is present on the server, whereas less frequently accessed files are tiered to the cloud. This is desirable for those files that you are not using very often but you still want them to be around.
If the file server becomes unavailable for any reason in your datacenter or branch office, you just need to install the Sync agent on another server or virtual machine, and your users and applications can access the file again within minutes.
One of the great features of Azure File Sync is the integration with the Azure Backup service. Azure Backup is an Azure-based service that you can use to back up (or protect) and restore your data in the Microsoft cloud. Azure Backup replaces your existing on-premises and off-site backup solution with a cloud-based solution that is reliable, secure, and cost-competitive. For nearly every Azure File Sync deployment that I work on, I’ve seen that Azure Backup is considered as a requirement to provide an additional layer of security and protection.
In the first blog of this 3 part series, I discussed how Azure Backup integrates with Azure File Sync. Please make sure to check it before you continue reading this article. In the second part, I will share with you the new features and improvements that were added for Azure Backup and Azure Files (Sync) as of May 2020.
Azure Backup for Azure Files (Sync)
The long waiting support for Azure Files backup is over, Microsoft just announced on April 29, 2020, that Azure Backup for Azure Files/File Share/File Sync is now generally available (GA). You can read about the announcement here. At the time of this writing, not all regions are GA yet, but the support is coming very soon for all Azure regions. The list of the currently supported regions is documented in the support matrix list here.
Let’s now look at the enhancement and new features introduced to Azure Backup and Azure Files (Sync) GA.
Long-Term Backup Retention
If you have been using Azure File shares backup, you know that Microsoft initially supported only a daily backup policy for a maximum of 180 days (6 months) in the Azure Portal.
After extensive testing on making sure the snapshots are durable for long-term retention, Microsoft enabled on-demand backup that can retain your snapshots for 10 years using PowerShell. I have developed a tool to automate the backup for Azure File Shares using PowerShell so you can schedule it to take snapshots at regular intervals every week, month, or year for long-term retention.
The good news is, you can now create a long-term backup policy (Daily/Weekly/Monthly/Yearly) for Azure Files directly in the Azure Portal without the need to use PowerShell.
Now as part of the daily (mandatory) backup policy, you can now configure 200 days of daily backup instead of 180 days previously. Why only 200 days? Read on…
Azure Backup supports only 200 snapshots per file share limit at any given point in time. This is the current restriction of the platform to make sure your snapshots are durable. You could, for example, schedule 190 days as a daily backup, and 10 years as a yearly backup (190+10=200 snapshots), and so on and so forth…
Additionally, Microsoft has added on-demand snapshots (backup) so you can take up to ten times backups per day (manually triggered), as well as one daily scheduled backup, so a total of 11 snapshots per day you can take ONLY.
The reason behind this restriction is to protect you from running out of snapshots support which is 200. And when you try to trigger more than 10 manual backups per day, you will receive the following error. The Error Message below will be updated to reflect the current limitation.
Alerts and reporting
By integrating Azure Backup with Azure File Sync, you also get a detailed report of Azure Files (Sync) backup by leveraging the power of Azure Backup reporting capability, for this to work, you need to make sure that your Recovery Services Vaults (Diagnostics settings) have been configured to send data to Log Analytics as shown below.
Then you can simply navigate to your vault and click on the Backup Reports menu item. As of today, under Summary you will see only how many Protected Instances you are using for Azure Files since the backup is NOT transferred to the Recovery Services Vaults. In other words, there is no Cloud Storage consumption from the backup perspective for Azure Files (Sync).
There are few additional metrics you can see in the reporting blade related to Azure Files such as how many backup Jobs have succeeded and failed, how many backup items corresponding to the file share type.
For alerts, you will also get critical and warning alerts related to your Azure Files (Sync) backup and restore as follows:
- Critical: In principle, any backup or recovery failure (scheduled or user-triggered) would lead to the generation of an alert and would be shown as a Critical alert and also destructive operations such as delete backup.
- Warning: If the backup operation succeeds but with few warnings, they are listed as Warning alerts.
- Informational: As of today, no informational alert is generated by the Azure Backup service for Azure Files.
Protection against accidental deletion
On May 27, 2020, the Azure Backup team in coordination with the Azure Files team announced the general availability of the long-awaited feature “Soft Delete” to protect your file share(s) from accidental deletion and malicious actor.
Soft delete protects your Azure file shares from accidental deletion. Soft delete acts like a recycle bin for Azure file shares, meaning that deleted shares remain recoverable for their entire retention period (7 days by default for storage accounts created after January 31st, 2021).
You will be charged for soft deleted data on the snapshot meter. If you have automated the creation of new storage accounts and the creation/deletion of new file shares within them, you must modify your scripts to explicitly disable soft delete after the creation of a new storage account.
Soft delete will remain disabled by default for existing storage accounts.
What is Soft Delete?
Soft delete acts like a recycle bin for your file shares, protecting your Azure file shares from accidental deletion. Now when a file share is deleted, it transitions to a soft-deleted state in the form of a soft deleted snapshot. You get to configure how long soft deleted data is recoverable before it is permanently erased.
Soft Delete is a new feature that enables intermediate state “Soft deleted state” for file share(s) when they are deleted accidentally or maliciously. The soft delete feature is configurable at the Storage account level but works only at the File share level. In other words, when you enable soft delete at the storage account level, then all the existing file shares, as well as the newly created ones will be protected and adhere to this policy. When soft delete is enabled, you need to define the retention policy (period) in days, the retention policy determines the time window for which file share contents would be retained before permanent deletion.
Soft Delete for file share can be enabled at existing storage accounts OR you can enable it during the creation of storage account as shown in the image below.
Please do not confuse between Blob soft delete and File share soft delete, both features are enabled at the storage account level but they serve different purposes.
How Azure Backup leverages Soft Delete?
The high-level overview of how Azure Backup leverages Azure File share with soft delete is illustrated in the following diagram:
When you configure Azure Backup for Azure File Share, what will happen is the following:
- You create or select an existing Recovery Services Vaults.
- You enable backup and select Azure File Share.
- Then you select the source Azure storage account where the Azure File share(s) reside. Then the storage account will be registered with the Recovery Services Vaults (at the time of this writing, no backup is transferred to the Recovery Services Vaults).
- The next step is to select one or more file share(s) which you want to protect.
- Azure Backup enables “Soft Delete” on the storage account with the default retention period of 14 days.
- Azure Backup enables “Delete Lock” on the protected storage account level.
- Last but not least, you can choose or create a new backup policy (daily, weekly, monthly, or yearly backup up to 10 years only through the Azure Portal).
- Finally, when each backup job runs, the Azure Backup service will ensure that the soft delete is always turned on.
At the time of this writing (May 28th, 2020), Azure Backup supports for Soft Delete feature is active in West Central US and is being rolled out for other Azure regions as well. Soft delete is supported only for standard and premium storage accounts and is currently enable from the Azure Backup side, please check the current supported regions here.
Please note that you can also reset the retention period settings as per your requirement (1-day => to 365 days). However, Azure Backup will always set the retention (soft delete) policy to 14 days if you set it below 14. On the other hand, if the retention policy is greater than 14 days (i.e. 15…365), Azure Backup will adhere to this policy and keep it as is.
This retention policy determines the time window you’ll have to recover your file share contents and snapshots after any accidental delete operation. The recovery points are preserved during this duration and once you undelete the file share, backups start running successfully with no additional configuration needed from your side.
When you want to restore your data from a deleted file share, you want to undelete the file share first within the retention period. If no action has been taken within the retention period, then the snapshots and file share contents are permanently deleted.
To learn more on how to enable and disable Soft Delete for Azure File Shares via REST APIs to automate that process, please check the following step-by-step guide.
What about billing during the retention period?
When a file share(s) is in the deleted state, you will be charged for the snapshots storage, as well as for the protected instance cost (more details on this in the next section). The price will be based on used capacity charged at the snapshots rate whether you are using standard or premium file shares.
On August 7, 2020, the Azure Backup team has simplified the experience of configuring backup for Azure file shares by giving users the ability to seamlessly enable backup in two simple steps from the file share blade directly.
Browse to your file shares and select the desired file share you want to back up. Select Backup under the Operations section of the file share pane. The Configure backup pane will load on the right as shown in the below figure. Select the desired Recovery services vault or create a new one, then choose or create a new backup policy, and finally Click ‘Enable Backup‘ to start protecting the file share.
Once the backup is enabled on the file share, you can also restore from the file share blade directly without going to the Recovery Services vaults. Browse to your file shares and select the desired file share you want to back up. Select Backup under the Operations section of the file share pane. Select ‘Restore Share‘ if you want to restore a full Azure file share, or restore individual files or folders by selecting ‘File Recovery‘ as shown in the below figure. You can also run an on-demand backup by clicking ‘Backup now‘.
With the integration of Azure Backup into the file share management blade, you can now perform the backup and restore related operations directly from the file share blade.
Azure Backup Pricing for Azure Files
As for the pricing details, backup for Azure Files (Sync) is calculated as follows:
- Azure Backup uses Azure File Share snapshots for creating recovery points. Storage charges incurred for snapshots are billed along with your Azure Files usage as documented here.
- The combined size of all backed-up Azure File Shares in a Storage Account determines the instance size while using Backup for Azure Files. Below is the updated price for the protect instance size based on the West Central US region. The Azure Backup pricing calculator is being updated to reflect all Azure regions.
|SIZE OF EACH INSTANCE||AZURE BACKUP PRICE PER MONTH|
|Instance is > 250 GB||$5|
|Instance||60% of Azure Files Protected instances price per month|
Let’s take the following real-world example:
- The total size of all Azure File Shares that are going to be backed up in the same storage account is 2 TB (2,048 GB).
- The daily churn rate is moderate ~ 3%. The churn is the amount of new data every day (that is, written or appended to existing files).
- The redundancy of the original file share (where snapshots would be stored) in the storage account is Locally-Redundant Storage (LRS).
- The type of Storage Account used in this example is Standard. At the time of this writing, we have two types of storage account for file shares (Standard/Premium). However, Microsoft announced at Ignite 2019 that a new type of storage Tier is coming very soon to reduce the storage cost of your Azure Files which is known as “Transaction Optimized“, “Hot“, and “Cool” tiers. The new storage tiers are announced by Microsoft, please check the following article for more details.
- The Azure Backup policy definition selected for this example as follows:
- Daily: 30 Days.
- Weekly: 4 Weeks.
- Monthly: 6 Months.
- Yearly: 5 Years.
The estimated monthly and yearly price will be calculated as follows:
As of this writing, the combined size (point 2) for the protected instance will not be chargeable until the first of August 2020. The pricing will also be dropped (reduced) as soon as Microsoft announces the new storage tiers. Stay Tuned!
Please note that Azure Files (Sync) pricing is separate from Azure Backup, you need to take into consideration the following 4 additional charges: Sync Server price (1st server is free), data storage price, operations and data transfer price, and outbound (network) data transfers price.
When designing a backup strategy for Azure file shares, you need to take into consideration the following two constraints:
- All the file shares hosted in the same storage account need to be protected with the same Recovery Services vaults.
- The maximum number of file shares that can be protected per vault is 2000, and the maximum number of storage accounts that can be registered per vault is 200.
I worked with several customers that usually choose Recovery Services vaults in such a way to associate certain business-related demarcations, similar to Resource Groups. That’s also a viable option.
Ideally, any configuration or deployment that fulfills the above constraints should be good to go and you don’t have to worry about any scalability, soft/hard limits per vault or subscription, and potential I/O bottlenecks.
There are a lot of improvements coming to Azure Backup and Azure Files (Sync). Microsoft is currently working on a new set of features:
- Transferring (hardening) to Recovery Services Vault instead of relying on keeping the share snapshots within the same file share/storage account.
- Soft-Delete – Protecting against accidental deletion of your Azure file share.
- Protecting against accidental deletion of an individual (specific) snapshot. Please note that deleting a specific snapshot will NOT break the backup/restore process, you will lose only the recovery point for that specific point in time snapshot.
- Configure backup directly from the Azure File share blade (UI) instead of going to the Recovery Services Vault.
- Schedule multiple snapshots per day using Azure Backup Policy. (Today is 1 snapshot).
- Take weekly/monthly full copies of File Share to Recovery Services Vault.
- Use snapshots replicated to a paired region to restore File share in the paired region.
- Take daily/weekly/monthly incremental backups of File shares to Recovery Services Vault?
The Azure Backup team needs your help to prioritize those features, please take the quick survey here and help to prioritize those features.
Azure File Sync extends on-premises file servers into Azure providing cloud benefits while maintaining performance and compatibility. Azure File Sync provides:
- Multi-site access – provide write access to the same data across Windows servers and Azure Files.
- Cloud tiering – store only recently accessed data on local servers.
- Integrates with Azure backup – no need to back up your data on-premises.
- Fast disaster recovery – restore file metadata immediately and recall data as needed.
By integrating Azure Backup with Azure File Sync, you will save a lot of storage management and reduce operational and licensing costs.
This is the current enhancement of Azure Backup integration with Azure Files (Sync), please make sure to check Part III which is planned to be released by the end of summer 2020 where I will show you the great improvements that are coming to Azure Backup and Azure Files (Sync).
I hope you find this quick guide useful. To learn more about Azure File Sync, please check the following articles.
Thank you for reading my blog.
If you have any questions or feedback, please leave a comment.