Sync and Backup Synology NAS to Azure Storage

7 Min. Read

Syncing and backing up data between on-premises locations and the public cloud have become a very common practice for many organizations, and sometimes for small to medium businesses and even for home users. We have seen organizations using solutions from storage providers to sync and migrate data to Azure Storage and other cloud providers. Additionally, to prevent hardware failure, and be able to access files while away from the office, we’ve also seen organizations wanting to migrate data from on-premises to Microsoft Azure.

In this article, we would like to share our experience on how to sync and backup Synology NAS to Microsoft Azure Storage for data migration.

Introduction

We have recently migrated a full Synology RS820RP+ to Microsoft Azure. Synology NAS devices are generally used by home users and small to medium businesses, it offers a very cost-effective way to migrate and synchronize files to cloud platforms. The configuration is pretty easy and the synchronization is pretty effective based on our experience.

With Cloud Sync, you can seamlessly sync and share files among your Synology NAS and multiple public clouds, such as Dropbox, Microsoft Azure, Baidu Cloud, and Google Drive, etc.

For the remainder of this article, we will use Azure storage as our backup and migration target.

Prerequisites

To follow this article, you need to have the following:

1) Azure subscription – If you don’t have an Azure subscription, you can create a free one here.

2) An Azure storage account – To create a general-purpose v2 storage account, you can follow the instructions described here.

3) You also need to create at least one container in the storage account. Optionally, optimize storage costs by automatically managing the data lifecycle (more on this in the next section).

4) Synology NAS which supports Cloud Sync – Check the official documentation from Synology and verify if your NAS model is listed.

5) Synology Cloud Sync add-on package (more on this in the next section).

6) Optimize cloud synchronization performance (more on this in the next section).

Install Cloud Sync

Synology Cloud Sync is extremely easy to configure. Once you’ve logged in to the web portal of your Synology NAS, it can be found in the Package Center, you can install it with one click (and follow the wizard).

Once we installed the add-on package from the “Package Center” as shown in the figure below, we can start creating sync jobs.

Install Synology Cloud Sync
Install Synology Cloud Sync

As mentioned in the introduction, Cloud Sync supports many public cloud providers such as Microsoft Azure, OneDrive, OneDrive for Business, AWS S3, GCP Cloud Storage, Google Drive, Dropbox, etc.

Azure Storage Account

Assuming you have already created a general-purpose v2 storage account as described in this article.

The next step is to configure the storage account network firewall to accept connections from specific IP addresses (network) for added security.

Navigate to Networking under the Security + networking blade and then click on “Selected networks“.

If your storage account is connected to an existing virtual network (VNet), you can add it here or create a new VNet if you want (this is useful for private endpoint scenarios). The most important point is to add the public IP address of the organization to the firewall rule so the NAS device can reach it, and your client IP address as shown in the figure below (we have blur-boxed the value for obvious reasons). Click Save.

The firewall settings allowing access to storage services will remain in effect for up to a minute after saving updated settings restricting access.

Azure Storage firewalls and virtual networks
Azure Storage firewalls and virtual networks

This restricts access to the storage account only from the company on-premises networks (public endpoint), or from the VNet it connects to (private endpoint), and other Azure services (since we’ve selected “Allow Azure services on the trusted services list to access this storage account” under Exceptions.

Storage firewall rules apply to the public endpoint of a storage account. You don’t need any firewall access rules to allow traffic for private endpoints of a storage account. The process of approving the creation of a private endpoint grants implicit access to traffic from the subnet that hosts the private endpoint.

The next step is to create one or more blob containers to be used as sync/backup targets by Synology Cloud Sync.

Navigate to Containers under the Data storage blade and then click on “+ Container” as shown in the figure below.

Create a blob container
Create a blob container

Last, an optional step is to optimize storage costs by automatically managing the data lifecycle. If you want to migrate the data from the NAS and archive it in Azure which could be rarely accessed afterward, then you could leverage the Archive tier to reduce costs.

Lifecycle management offers a rich, rule-based policy for general-purpose v2 and blob storage accounts. You could use the policy to transition your data to the appropriate access tiers (Hot, Cool, or Archive), or expire at the end of the data’s lifecycle.

In this example, we will move all block blobs from a specific container to the archive tier after 60 days.

Navigate to the Lifecycle management under the Data management blade and then click on “Code View” as shown in the figure below.

Lifecycle management
Lifecycle management

Paste the following code and then click Save. Make sure to adjust the “prefixMatch” to match your container’s name and the number of days that you want to set.

Some data stays idle in the cloud and is rarely, if ever, accessed. The following lifecycle policy is configured to archive data after it is ingested with 60 days. This example transitions block blobs in a container named “azure-nas-production01” and “azure-nas-archive01” into an archive tier. The transition is accomplished by acting on blobs 60 days after the last modified time.

{
  "rules": [
    {
      "enabled": true,
      "name": "MoveToArchiveRule",
      "type": "Lifecycle",
      "definition": {
        "actions": {
          "baseBlob": {
            "tierToArchive": {
              "daysAfterModificationGreaterThan": 60
            }
          }
        },
        "filters": {
          "blobTypes": [
            "blockBlob"
          ],
          "prefixMatch": [
            "azure-nas-production01",
            "azure-nas-archive01"
          ]
        }
      }
    }
  ]
}

Configure Cloud Sync

In this section, you need to configure the Cloud Sync by creating one or more sync jobs.

Launch the Synology NAS web portal, and then open the Cloud Sync package. On the bottom left-hand side, click the plus (+) sign and then choose “Azure storage” as shown in the figure below. Click Next to continue.

Cloud Sync - Azure Storage
Cloud Sync – Azure Storage

Enter the following details and then click Next to continue.

> Service endpoint: Azure Global
> Storage account: <storage account name>
> Access key: the primary or secondary key for the storage account
> Blob container name: <blob container name>

Azure Cloud Storage Settings
Azure Cloud Storage Settings

Then choose the local path (a share on the NAS), and the remote path in Azure storage blob (Root folder), you can also select the sync direction. In this case, we’ve chosen Upload local changes only (since we need to replicate the data one-way from the NAS to Azure blob storage, this is useful for data migration), you can select Bidirectional (so changes from both ends will be replicated to each other), or Download remote changes only (so the data will replicate one-way from Azure blob storage to the NAS). Click Next to continue.

Task Settings
Task Settings

You can also set a schedule as shown in the figure below (E.g. Stop sync during working hours to save network bandwidth). Click OK and then click Next to continue.

Schedule Settings
Schedule Settings

Please note that you can still modify these settings after the jobs are created under the “Settings” and “Task List” tab as shown in the figure below. For example, you can configure the polling period, set network throttling, folder exclusions, file filter (based on file extensions), add/remove/modify sync folders, don’t sync files over X number of MB, change sync direction, etc. The polling period value is important for the Bidirectional sync direction (so the NAS can poll the changes from the storage account container, in this example, it’s set to 60 seconds).

Modify Job settings
Modify Job settings

Depending on the size of the folder and your Internet link speed, the initial synchronization can take a while. Once completed, you’ll see the status as “Up to date” as shown in the figure below.

Cloud Sync - Up to date
Cloud Sync – Up to date

At this stage, any changes on the NAS folder or the blob container will be replicated. You can see the event type under the “History” tab as shown in the figure below. In this example, we are uploading only to Azure storage (one way).

History job
History job

Cloud Sync Settings and Performance

By default, the Cloud Sync concurrent uploads and downloads are set to 3. So if you have a good internet connection, then the synchronization and backup from your NAS will take a long time to complete.

Internet Speed Test
Internet Speed Test

To increase the sync and backup performance, open the Synology Cloud Sync app and then click the “Settings” button as shown in the figure below.

Cloud Sync Settings
Cloud Sync Settings

On the “Concurrent uploads/downloads” set it to 20 as shown in the figure below. This will increase the number of concurrent performance file uploads/downloads to improve the overall synchronization performance.

Additionally, the “History” log can only show 20,000 items by default. You could also increase the “Maximum records” from 20,000 to the maximum supported (100,000) records to increase the history log for each linked account.

Sync and Backup Synology NAS to Azure Storage 1

The default concurrent uploads and downloads are 3, which gives you about 4 MBps. once set to 7, we got around 8 MBps, and setting it to 20, we got a nice steady 15 MBps.

On the other hand, the container in the storage account under (Properties | Calculate size) will count up to 20,000 items and 2.41 GiB only as shown in the figure below.

Container size
Container size

If you want to calculate the capacity size for a specific container, then you could run the following PowerShell script after you connect to your Azure subscription.

# Connect to Azure
Connect-AzAccount

# Storage account to be used - change to match your resources
$resourceGroup = "arg-weu-nas-prd01"
$storageAccountName = "asabloblistnas01"
$containerName = "azure-nas-archive01"

# Get a reference to the storage account and the context
$storageAccount = Get-AzStorageAccount `
  -ResourceGroupName $resourceGroup `
  -Name $storageAccountName
$ctx = $storageAccount.Context 

# Get a list of all of the blobs in the container 
$listOfBlobs = Get-AzStorageBlob -Container $ContainerName -Context $ctx 

# Zero out our total
$length = 0

# Loops through the list of blobs and retrieves the length for each blob
#   and adds it to the total
$listOfBlobs | ForEach-Object {$length = $length + $_.Length}

# Output the blobs and their sizes and the total 
Write-Output "List of Blobs and their size (length)"
Write-Host " " 
$listOfBlobs | select Name, Length
Write-Host " "
Write-Output "Total Length = " $length

You could also look at the storage account (storage browser) experience in the Azure Portal and see the total number of blobs and the total data stored as shown in the figure below. In this example, we have 4.96 Million blobs and the total size is 6.63 TiB.

"Storage

That’s it there you have it!

Conclusion

In this article, we showed you how to sync and backup Synology NAS to Microsoft Azure Storage to prevent hardware failure, and be able to access files while away from the office. This will also be useful in data migration scenarios to Azure storage and decommission on-premises NAS.

At the time of this writing, Synology Cloud Sync supports only 2 Azure environments (Azure Global and Azure China). It doesn’t support other environments such as Azure Germany, or Azure Government.

Cloud Sync Service endpoint
Cloud Sync Service endpoint

Cloud Sync does not support Azure Files, only blob storage. This limit prevents people who need to access the files on the Storage accounts via SMB.

To sync to Azure Storage account, it uses Storage Account access keys. Some organizations prohibit users from using access keys and disabling the storage account key access. It would be great if we can use Azure AD Service Principals that have sufficient Role-Based Access Control (RBAC) permissions to access the storage account.

Do you want to learn more about Azure Storage including Azure Blobs and Azure File Shares? Make sure to check my recently published online course here: Azure Storage Essential Training.

__
Thank you for reading my blog.

If you have any questions or feedback, please leave a comment.

-Charbel Nemnom-

Related Posts

Previous

Get the Most Out of Windows Admin Center – Second Edition

Manage Security Content as Code with Microsoft Sentinel

Next

Let me know what you think, or ask a question...

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Subscribe to Stay in Touch

Never miss out on your favorite posts and our latest announcements!

The content of this website is copyrighted from being plagiarized!

You can copy from the 'Code Blocks' in 'Black' by selecting the Code.

Please send your feedback to the author using this form for any 'Code' you like.

Thank you for visiting!