Sync and Backup Synology NAS to Azure Storage

9 Min. Read

Syncing and backing up data between on-premises locations and the public cloud have become a very common practice for many organizations, and sometimes for small to medium businesses and even for home users. We have seen organizations using solutions from storage providers to sync and migrate data to Azure Storage and other cloud providers. Additionally, to prevent hardware failure, and be able to access files while away from the office, we’ve also seen organizations wanting to migrate data from on-premises to Microsoft Azure.

In this article, we would like to share our experience on how to sync and backup Synology NAS to Microsoft Azure Storage for data migration.

Introduction

We have recently migrated a full Synology RS820RP+ to Microsoft Azure. Synology NAS devices are generally used by home users and small to medium businesses, it offers a very cost-effective way to migrate and synchronize files to cloud platforms. The configuration is pretty easy and the synchronization is pretty effective based on our experience.

With Cloud Sync, you can seamlessly sync and share files among your Synology NAS and multiple public clouds, such as Dropbox, Microsoft Azure, Baidu Cloud, Google Drive, etc.

For the remainder of this article, we will use Azure storage as our backup and migration target.

Prerequisites

To follow this article, you need to have the following:

1) Azure subscription – If you don’t have an Azure subscription, you can create a free one here.

2) An Azure storage account – To create a general-purpose v2 storage account, you can follow the instructions described here.

3) You also need to create at least one container in the storage account. Optionally, optimize storage costs by automatically managing the data lifecycle (more on this in the next section).

4) Synology NAS which supports Cloud Sync – Check the official documentation from Synology and verify if your NAS model is listed.

5) Synology Cloud Sync add-on package (more on this in the next section).

6) Optimize cloud synchronization performance (more on this in the next section).

Install Cloud Sync

Synology Cloud Sync is extremely easy to configure. Once you’ve logged in to the web portal of your Synology NAS, it can be found in the Package Center, you can install it with one click (and follow the wizard).

Once we installed the add-on package from the “Package Center” as shown in the figure below, we can start creating sync jobs.

Install Synology Cloud Sync
Install Synology Cloud Sync

As mentioned in the introduction, Cloud Sync supports many public cloud providers such as Microsoft Azure, OneDrive, OneDrive for Business, AWS S3, GCP Cloud Storage, Google Drive, Dropbox, etc.

Azure Storage Account

Assuming you have already created a general-purpose v2 storage account as described in this article.

The next step is to configure the storage account network firewall to accept connections from specific IP addresses (network) for added security.

Navigate to Networking under the Security + networking blade and then click on “Selected networks“.

If your storage account is connected to an existing virtual network (VNet), you can add it here or create a new VNet if you want (this is useful for private endpoint scenarios). The most important point is to add the public IP address of the organization to the firewall rule so the NAS device can reach it, and your client’s IP address as shown in the figure below (we have blur-boxed the value for obvious reasons). Click Save.

The firewall settings allowing access to storage services will remain in effect for up to a minute after saving updated settings restricting access.

Azure Storage firewalls and virtual networks
Azure Storage firewalls and virtual networks

This restricts access to the storage account only from the company on-premises networks (public endpoint), or from the Virtual Network (VNet) it connects to (private endpoint), and other Azure services (since we’ve selected “Allow Azure services on the trusted services list to access this storage account” under Exceptions.

For more information on how to connect with Private Endpoint for added security, please check the following guide.

Storage firewall rules apply to the public endpoint of a storage account. You don’t need any firewall access rules to allow traffic for private endpoints of a storage account. The process of approving the creation of a private endpoint grants implicit access to traffic from the subnet that hosts the private endpoint.

The next step is to create one or more blob containers to be used as sync/backup targets by Synology Cloud Sync.

Navigate to Containers under the Data storage blade and then click on “+ Container” as shown in the figure below.

Create a blob container
Create a blob container

Last, an optional step is to optimize storage costs by automatically managing the data lifecycle. If you want to migrate the data from the NAS and archive it in Azure which could be rarely accessed afterward, then you could leverage the Archive tier to reduce costs.

Lifecycle management offers a rich, rule-based policy for general-purpose v2 and blob storage accounts. You could use the policy to transition your data to the appropriate access tiers (Hot, Cool, or Archive), or expire at the end of the data’s lifecycle.

In this example, we will move all block blobs from a specific container to the archive tier after 60 days.

Navigate to the Lifecycle management under the Data management blade and then click on “Code View” as shown in the figure below.

Lifecycle management
Lifecycle management

Paste the following code and then click Save. Make sure to adjust the “prefixMatch” to match your container’s name and the number of days that you want to set.

Some data stays idle in the cloud and is rarely, if ever, accessed. The following lifecycle policy is configured to archive data after it is ingested within 60 days. This example transitions block blobs in a container named “azure-nas-production01” and “azure-nas-archive01” into an archive tier. The transition is accomplished by acting on blobs 60 days after the last modified time.

{
  "rules": [
    {
      "enabled": true,
      "name": "MoveToArchiveRule",
      "type": "Lifecycle",
      "definition": {
        "actions": {
          "baseBlob": {
            "tierToArchive": {
              "daysAfterModificationGreaterThan": 60
            }
          }
        },
        "filters": {
          "blobTypes": [
            "blockBlob"
          ],
          "prefixMatch": [
            "azure-nas-production01",
            "azure-nas-archive01"
          ]
        }
      }
    }
  ]
}

Configure Cloud Sync

In this section, you need to configure the Cloud Sync by creating one or more sync jobs.

Launch the Synology NAS web portal, and then open the Cloud Sync package. On the bottom left-hand side, click the plus (+) sign and then choose “Azure storage” as shown in the figure below. Click Next to continue.

Cloud Sync - Azure Storage
Cloud Sync – Azure Storage

Enter the following details and then click Next to continue.

> Service endpoint: Azure Global
> Storage account: <storage account name>
> Access key: the primary or secondary key for the storage account
> Blob container name: <blob container name>

Azure Cloud Storage Settings
Azure Cloud Storage Settings

Then choose the local path (a share on the NAS), and the remote path in the Azure storage blob (Root folder), you can also select the sync direction. In this case, we’ve chosen Upload local changes only (since we need to replicate the data one-way from the NAS to Azure blob storage, this is useful for data migration), you can select Bidirectional (so changes from both ends will be replicated to each other), or Download remote changes only (so the data will replicate one-way from Azure blob storage to the NAS). Click Next to continue.

Task Settings
Task Settings

You can also set a schedule as shown in the figure below (E.g. Stop sync during working hours to save network bandwidth). Click OK and then click Next to continue.

Schedule Settings
Schedule Settings

Please note that you can still modify these settings after the jobs are created under the “Settings” and “Task List” tab as shown in the figure below. For example, you can configure the polling period, set network throttling, folder exclusions, file filter (based on file extensions), add/remove/modify sync folders, and don’t sync files over X number of MB, change sync direction, etc. The polling period value is important for the Bidirectional sync direction (so the NAS can poll the changes from the storage account container, in this example, it’s set to 60 seconds).

Modify Job settings
Modify Job settings

Depending on the size of the folder and your Internet link speed, the initial synchronization can take a while. Once completed, you’ll see the status as “Up to date” as shown in the figure below.

Cloud Sync - Up to date
Cloud Sync – Up to date

At this stage, any changes on the NAS folder or the blob container will be replicated. You can see the event type under the “History” tab as shown in the figure below. In this example, we are uploading only to Azure storage (one way).

History job
History job

Cloud Sync Settings and Performance

By default, the Cloud Sync concurrent uploads and downloads are set to 3. So if you have a good internet connection, then the synchronization and backup from your NAS will take a long time to complete.

Internet Speed Test
Internet Speed Test

To increase the sync and backup performance, open the Synology Cloud Sync app and then click the “Settings” button as shown in the figure below.

Cloud Sync Settings
Cloud Sync Settings

On the “Concurrent uploads/downloads” set it to 20 as shown in the figure below. This will increase the number of concurrent performance file uploads/downloads to improve the overall synchronization performance.

Additionally, the “History” log can only show 20,000 items by default. You could also increase the “Maximum records” from 20,000 to the maximum supported (100,000) records to increase the history log for each linked account.

Sync and Backup Synology NAS to Azure Storage 1

The default concurrent uploads and downloads are 3, which gives you about 4 MBps. once set to 7, we got around 8 MBps, and setting it to 20, we got a nice steady 15 MBps.

On the other hand, the container in the storage account under (Properties | Calculate size) will count up to 20,000 items and 2.41 GiB only as shown in the figure below.

Container size
Container size

If you want to calculate the capacity size for a specific container, then you could run the following PowerShell script after you connect to your Azure subscription.

# Connect to Azure
Connect-AzAccount

# Storage account to be used - change to match your resources
$resourceGroup = "arg-weu-nas-prd01"
$storageAccountName = "asabloblistnas01"
$containerName = "azure-nas-archive01"

# Get a reference to the storage account and the context
$storageAccount = Get-AzStorageAccount `
  -ResourceGroupName $resourceGroup `
  -Name $storageAccountName
$ctx = $storageAccount.Context 

# Get a list of all of the blobs in the container 
$listOfBlobs = Get-AzStorageBlob -Container $ContainerName -Context $ctx 

# Zero out our total
$length = 0

# Loops through the list of blobs and retrieves the length for each blob
#   and adds it to the total
$listOfBlobs | ForEach-Object {$length = $length + $_.Length}

# Output the blobs and their sizes and the total 
Write-Output "List of Blobs and their size (length)"
Write-Host " " 
$listOfBlobs | select Name, Length
Write-Host " "
Write-Output "Total Length = " $length

You could also look at the storage account (storage browser) experience in the Azure Portal and see the total number of blobs and the total data stored as shown in the figure below. In this example, we have 4.96 Million blobs and the total size is 6.63 TiB.

"Storage

Access Synology NAS in Azure Storage

Now that you have your data from the NAS synced to Azure Storage, you can access the data in several different ways.

You could use the Azure portal, Azure Resource Manager APIs, PowerShell, Azure CLI, or by using the Azure Storage Explorer.

For the purpose of this article, we will use the Azure Storage Explorer which you can download for free from here and install on your machine based on your operating system (Windows, macOS, or Linux).

Once Azure Storage Explorer is installed on your machine, launch it and then click on the plug-in icon from the left-hand side as shown in the figure below.

Microsoft Azure Storage Explorer
Microsoft Azure Storage Explorer

Next, select Subscription to sign in to Azure to access storage resources such as (Blob container).

Connect to Azure Storage
Connect to Azure Storage

Next, select which Azure environment will you use to sign in?

At the time of this writing, Synology Cloud Sync supports only 2 Azure environments (Azure and Azure China). In this example, we are using Azure global. Click Next to sign in as shown in the figure below.

Your browser should now open and redirect you to a login page. Enter your Azure AD account details and approve the MFA.

Select Azure Environment
Select Azure Environment

Once you are authenticated. You can close the browser and return to the Azure Storage Explorer application.

Select again the plug-in icon from the left-hand side and this time choose the Blob container option as shown in the figure below.

Attach to an individual Blob container
Attach to an individual Blob container

Next, select how will you connect to the blob container? The recommended option and security best practice is to sign in using Azure Active Directory (Azure AD).

As a side note, before using the Azure AD option to connect, make sure that you’ve switched the authentication method for your Azure blob container from Access Key to Azure AD User Account. Check this guide for more details.

In this example, we are using Azure AD and the authentication method is already switched to Azure AD User Account. Click Next to continue.

Select Connection Method
Select Connection Method

Next, select your Azure account with access to the resource and make sure the right Tenant is selected at the bottom, then click Next to continue.

On the Enter Connection Info page, enter the full Blob container URL as shown in the example below:

If you are using the public endpoint and/or the private endpoint (recommended) of your storage account, then the full URL will look like the following. Please make sure to replace the <storageaccountname> and the <blobcontainername> with your own values.

https://<storageaccountname>.blob.core.windows.net/<blobcontainername>

We have blur-boxed the URL value for obvious reasons. Click Next to continue.

Enter Connection Info
Enter Connection Info

On the summary page, review all the settings that you have entered, and finally click Connect.

Back to the Azure Storage Explorer application, you will see that the Blob Container is attached as shown in the figure below. We have blur-boxed the content for obvious reasons.

You can download (read), upload (write), or delete data if you have the right permissions assigned to your Azure AD account.

Attach Blob Container to Azure Storage Explorer
Attach Blob Container to Azure Storage Explorer

As shown in the screenshot above, the blobs are in the Archive access tier to reduce storage costs. While a file is in the Archive tier, it can’t be read or modified immediately. To read or download a file from the Archive tier, you must first rehydrate (move) it to an online tier, either Hot or Cool.

You can right-click on the desired blob (file) that you want to access (read) and then select “Change Access Tier…“.

On the Update Access Tier page, select the desired access tier (Hot or Cool) and then select the desired Rehydration priority, and then click Apply.

Update Access Tier
Update Access Tier

Please note that the data in the Archive tier can take up to 15 hours to rehydrate (to become accessible), depending on the priority you specify below for the rehydration operation (Standard or High Priority). Learn more about access tiers.

  • Standard priority: The rehydration request will be processed in the order it was received and may take up to 15 hours.
  • High priority: The rehydration request will be prioritized over standard priority requests and may complete in less than one hour for objects under 10 GB in size.

Last but not least, if you have more than one container that you synced from Synology NAS to Azure storage, then you need to repeat the same steps described in this section to attach the second Blob Container to Azure Storage Explorer.

That’s it there you have it!

Conclusion

In this article, we showed you how to sync and backup Synology NAS to Microsoft Azure Storage to prevent hardware failure, and be able to access files while away from the office. This will also be useful in data migration scenarios to Azure storage and decommission on-premises NAS.

At the time of this writing, Synology Cloud Sync supports only 2 Azure environments (Azure Global and Azure China). It doesn’t support other environments such as Azure Germany, or Azure US Government.

Cloud Sync Service endpoint
Cloud Sync Service endpoint

Cloud Sync does not support Azure Files, only blob storage. This limit prevents people who need to access the files on the Storage accounts via SMB.

To sync to Azure Storage account, it uses Storage Account access keys. Some organizations prohibit users from using access keys and disabling the storage account key access. It would be great if we can use Azure AD Service Principals that have sufficient Role-Based Access Control (RBAC) permissions to access the storage account.

Do you want to learn more about Azure Storage including Azure Blobs and Azure File Shares? Make sure to check my recently published online course here: Azure Storage Essential Training.

__
Thank you for reading my blog.

If you have any questions or feedback, please leave a comment.

-Charbel Nemnom-

Related Posts

Previous

Get the Most Out of Windows Admin Center – Second Edition

Manage Security Content as Code with Microsoft Sentinel

Next

Let me know what you think, or ask a question...

The content of this website is copyrighted from being plagiarized!

You can copy from the 'Code Blocks' in 'Black' by selecting the Code.

Please send your feedback to the author using this form for any 'Code' you like.

Thank you for visiting!