You dont have javascript enabled! Please enable it! How To Backup Azure DevOps Git Repositories - CHARBEL NEMNOM - MVP | MCT | CCSP - Cloud & CyberSecurity

Effective Way To Backup Azure DevOps Repositories

13 Min. Read

Updated – 12/01/2024 – This article has been updated to include and back up Azure DevOps Git repositories from different projects. Please refer to the following section for more details.

Updated – 07/09/2023 – This article has been updated to include and back up Azure DevOps Git repositories within a project. Please refer to the following section for more details.

Updated – 24/08/2023 – This article has been updated to include the date and time of the Azure DevOps repo archive file, so all previous backup versions are kept and not overwritten.

Updated – 24/07/2023 – This article has been updated to use the system-generated token instead of the Personal Access Token (PAT) to improve the security and reliability of the Pipeline.

As a SaaS solution, Azure DevOps is highly available, and the instance data of an Organization is replicated to a partner region in the background for failover in case the primary region is down. When deleting a DevOps Organization it is in a soft-deleted state and can easily be restored in a 28-day grace period. Although code in Azure DevOps Repos is versioned by Source Control Management (SCM), e.g. Git, and Microsoft has an internal service backup with 28-day retention in place, this is not usable for customer-induced accidental deletions.

As such, a custom backup solution must be implemented to store data history outside of Azure DevOps. In this comprehensive guide, we will show you how to effectively automate Azure DevOps Backup Git repositories and then verify the restore operation.

Introduction

DevOps has been an emerging trend in the software development world for the past several years. While the term may be relatively new, it is a convergence of many practices that have been evolving for decades. DevOps is a revolutionary way to release software quickly and efficiently while maintaining a high level of security.

Azure DevOps — formerly known as Visual Studio Team Services (VSTS)— is a Software as a service (SaaS) platform from Microsoft that provides an end-to-end DevOps toolchain for developing and deploying software. It also integrates with most leading tools on the market and is a great option for orchestrating a DevOps toolchain. It also integrated perfectly with both Visual Studio and Visual Studio Code. While the code is perfectly safe on Azure infrastructure as committed by Microsoft here (Azure DevOps data protection overview), including periodic backups and Geo-replication, we do not have any control over it. And it does not prevent unintentional or malicious actions leading to data loss.

There are also cases where a centralized local backup of all projects and repositories is needed. These might include corporate policies, business continuity, and disaster recovery plans (BCDR).

Unfortunately, there is no out-of-the-box backup solution as confirmed by Microsoft here. However, what we could do is download the repository manually from Azure DevOps as a Zip file as shown in the figure below.

Effective Way To Backup Azure DevOps Repositories 1

But this may not be practical if we have a considerable amount of projects and repositories and need them backed up on a regular basis.

Let’s explore other methods and see how to automate the backup of the repositories.

Prerequisites

To follow this article, you need to have the following:

1) Azure subscription – If you don’t have an Azure subscription, you can create a free one here.

2) Azure storage account and container – To create a general-purpose v2 storage account, follow the instructions described here.

3) Azure DevOps – If you don’t have one, you can create a free one here.

  • You have an existing Azure Repos.
  • You are familiar with Azure Pipelines and YAML syntax.

4) Azure DevOps Service Connection – To create a new service connection, follow the steps described in this article. You can assign the scope at the Management Group or Azure Subscription Level.

Create Azure service connection in Azure DevOps
Create Azure service connection in Azure DevOps

Automate Azure DevOps backup repository

To automate the backup repository, you have a couple of different options as follows:

1) Programmatically back up source code from Azure DevOps using Azure DevOps REST API. Then you can use Azure Functions with a time trigger to run this on schedule.

2) Create a git bash script to mirror and back up the Git repository.

3) Clone the latest repository on your machine and then automate the backup using your favorite on-premises backup application such as SCDPM, Veeam, etc. OR,

4) You can leverage the Azure DevOps service to automate the backup and copy the repository to Azure Blob storage or to an Azure VM.

For the remainder of this article, we will use option 4 to automate the backup of the repository using the YAML Pipeline and command-line tasks. At a high level, the process looks like this:

1) Create an Azure DevOps Personal Access Token (PAT).

2) Create a YAML Pipeline including the following tasks:

  • Clone the Git repository.
  • Archive the Git repository.
  • Copy the archive file to Azure Blob storage.

Create an Azure DevOps Personal Access Token

Updated – 24/07/2023 – You can skip this section and move to the next section (Create a YAML Pipeline), you don’t need to create an Azure DevOps Personal Access Token (PAT). We’ll use the system-generated token instead of using (PAT) to improve the security and prevent the token from getting expired.

Sign in to the Azure DevOps portal and take the following steps:

On the top right corner of the portal, you have user settings next to your picture account. Clicking on it reveals the account menu where you can find the ‘Personal access tokens‘ option as shown in the figure below.

Effective Way To Backup Azure DevOps Repositories 2

The ‘Personal access tokens‘ page will open. Click on ‘New Token‘ to create one. For this example, we only need to check the ‘Read‘ for Code as shown in the figure below. Next, give a ‘Name‘ to the token, then set the ‘Expiration‘ date and click ‘Create‘.

Effective Way To Backup Azure DevOps Repositories 3

Once the token is created you will get the value, but since it won’t be shown again, you must copy and save it elsewhere. We will need this token in the next section to clone the repository.

Create a YAML Pipeline

Azure Pipelines supports continuous integration (CI) and continuous delivery (CD) to constantly and consistently test and build your code and ship it to any target. You accomplish this by defining a pipeline. And if there’s any update with your repository, the pipeline will be triggered to run and copy the entire repository.

To create a YAML pipeline, take the following steps:

1) Navigate to your team project on Azure DevOps in a new browser tab.

2) Navigate to the Pipelines hub on the left-hand side.

3) Click ‘New pipeline‘. We will use the wizard to automatically create the YAML definition based on our project.

Effective Way To Backup Azure DevOps Repositories 4

4) Next, select ‘Azure Repos Git‘ as the source hosting platform.

Effective Way To Backup Azure DevOps Repositories 5

5) Select your repository. In this example, the repo is called ‘infra-code-cn‘.

Effective Way To Backup Azure DevOps Repositories 6

6) Next, select the ‘Starter pipeline‘ template as the starting point for the backup pipeline.

Effective Way To Backup Azure DevOps Repositories 7

7) The starter pipeline will open including the following default YAML definition.

Effective Way To Backup Azure DevOps Repositories 8

8) Next, we need to update the YAML definition. The full YAML definition syntax to monitor and back up all branches looks like the following. Please refer to this section to get the latest version of the YAML definition file that includes a backup date and time for the archive file, so you will have all the previous backup versions of the files.

YAML definition syntax
YAML definition syntax

The YAML file includes the following logic:

  • Trigger (Continuous Integration): The wildcard (*) can monitor all your branches. So if there’s any update with your repository, the pipeline will be triggered to run.
  • Pool (Windows-latest): This task is written in PowerShell and thus works only when running on Windows agents.
  • Variables: We define the date and time the pipeline kicked off, so we can append it to the archive backup file name.
  • CmdLine Task: This command calls ‘git clone –mirror‘ to make a copy of your git repository. Here you need to use the personal access token that we created in the previous step. The System.AccessToken is a special variable that carries the security token used by the running build. The full syntax of the command line task looks like this:
steps:
  - task: CmdLine@2
  - bash: 'git clone --mirror https://${SYSTEM_ACCESSTOKEN}@dev.azure.com/{yourOrganizationName}/{yourProjectName}/_git/{yourRepoName}'
    env:
      SYSTEM_ACCESSTOKEN: $(System.AccessToken)
  • The Archive Files Task: This will take the git repository which was cloned from the previous step and then zipped to Date-Time_AzDevOpsBackup.zip and overwrite the existing archive file (if exists) on the VM image used in the Pool. The archive files task includes a range of standard archive formats such as .zip, .jar, .war, .ear, .tar, .7z, and more.
  • The File Copy Task: The copy task will take the archive Date-Time_AzDevOpsBackup.zip file and send it over to the Azure Blob storage. In this task, you need to specify the target service connection in Azure DevOps for the Azure subscription, storage account, and container name.

9) Finally, click Save and Run and then commit.

Effective Way To Backup Azure DevOps Repositories 9

10) Once you commit, you need to give the pipeline the needed permission to access the resource before it can run for the first time. Click ‘View‘.

Effective Way To Backup Azure DevOps Repositories 10

11) On the Waiting for Review page, click ‘Permit‘, and then click ‘Permit‘ on the confirmation access window.

Effective Way To Backup Azure DevOps Repositories 11

12) Once you permit, the pipeline will kick in and run the job. The job should be completed in about 2 minutes.

Effective Way To Backup Azure DevOps Repositories 12

13) You will also receive an email if the Build failed or succeeded similar to the one below.

Effective Way To Backup Azure DevOps Repositories 13

Verify the Backup

Now to verify the backup, sign in to the Azure Portal and browse to the storage account, then go to Containers under Blob service.

You will see a new container named ‘azure-devops-container‘ as we set in the YAML definition.

Effective Way To Backup Azure DevOps Repositories 14

Click on the container name and then open the folder which is the Blob Prefix (az-devops-backup), you will see the Backup.zip archive file as shown in the figure below.

Effective Way To Backup Azure DevOps Repositories 15

Restore from Backup

To restore from the backup, you need first to download the archive Date-Time_AzDevOpsBackup.zip blob from the storage container and extract it locally on your machine as shown in the figure below.

Download blob Azure DevOps Repo Backup.zip
Download blob Azure DevOps Repo Backup.zip

Once you extract the file, you will see the repository name with the .git extension. Open the command prompt window assuming you already have Git installed on your machine, browse to the extracted folder of the backup, and then run the following commands in sequence to push and restore your repository.

Please note that you can restore to the same project/repository or you can restore to a new project and new repository. If you want to restore to a new repository, make sure the new Git repository is NOT initialized, and then restore the backup using the following commands:

git checkout master
git remote remove origin
git remote add origin https://dev.azure.com/organization-name/project-name/_git/new-repo-name
git push -u origin --all
Restore Azure DevOps Repo from Backup
Restore Azure DevOps Repo from Backup

The “git push -u origin --all” command will ensure that all branches and other attributes are replicated into the new repo. The good news is, that once you restore the repository, it also brings along all the revision information (full-fidelity history) as shown in the figure below.

Effective Way To Backup Azure DevOps Repositories 16

That’s it there you have it!

Backup Azure DevOps YAML

You can find the latest version of the YAML pipeline below which will back up your Azure DevOps Git repository and keep your previous versions intact, in this case, the old backup versions won’t be overwritten, so you can restore from earlier backups if needed.

You can get a copy of this YAML file directly from my GitHub.

# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
# Date: 24-July-2023
# Update: 12-January-2024

trigger:
  branches:
    include:
    - '*'

pool:
  vmImage: 'windows-latest'

variables: 
 backup_date: $[format('{0:yyyy}-{0:MM}-{0:dd}T{0:HH}-{0:mm}-{0:ss}', pipeline.startTime)]

steps:
  - task: CmdLine@2
  - bash: 'git clone --bare https://${SYSTEM_ACCESSTOKEN}@dev.azure.com/{yourOrganizationName}/{yourProjectName}/_git/{yourRepoName}'      
    env:
      SYSTEM_ACCESSTOKEN: $(System.AccessToken)

  - task: ArchiveFiles@2
    displayName: 'Building archive file locally'
    inputs:
      rootFolderOrFile: .
      includeRootFolder: false
      archiveType: 'zip'
      archiveFile: '$(Build.ArtifactStagingDirectory)\$(backup_date)_AzDevOpsBackup.zip'
      replaceExistingArchive: true

  - task: AzureFileCopy@3
    displayName: 'Azure Blob File Copy'
    inputs:
      SourcePath: '$(Build.ArtifactStagingDirectory)\$(backup_date)_AzDevOpsBackup.zip'
      azureSubscription: 'AzDevOps-Bkp-Svc'
      Destination: 'AzureBlob'
      storage: 'storageaccountnamehere'
      ContainerName: 'azure-devops-container'
      BlobPrefix: 'az-devops-backup'
Backup Azure DevOps with date and time
Backup Azure DevOps with date and time

Backup Multiple Repos in Azure DevOps

A common question that keeps coming up is the following:

How can we apply the steps described above to include and back up multiple Azure DevOps repositories?

The good news is that I finally tackled this functionality, so now we can back up all or specific repos in a specific Azure DevOps project, or back up all or specific repos in different Azure DevOps projects. For this to work, please follow the steps described in this section.

First, you need to take the following YAML definition and update all the parameters marked with # CHANGE < very important.

This YAML file will be referenced later when we create the new pipeline. You can get a copy of this YAML file directly from my GitHub. Save this file in the desired repository and folder with the .yml extension:

# This is totally optional, you can remove all parameters and using hardcoded values in stages below
# Parameters are useful if you are running this pipeline manually and you need sometimes to pick only required components
parameters:
  - name: backupSourceCode
    displayName: 'Backup Azure DevOps Source Code'
    type: boolean
    default: true # Backup Azure DevOps Source Code (true/false)
  - name: backupVariableGroups
    displayName: 'Backup Azure DevOps Variable Groups'
    type: boolean
    default: false # Backup Azure DevOps Variable Groups (true/false)

trigger:
  none

stages:
  - stage: MAIN # CHANGE - the name of the stage, you can omit if you have only one stage and use jobs directly    
    jobs:
    - template: templates/multi-repo-backup-template.yml # CHANGE - the path and name of the backup template .yml file below
      parameters:        
        backupSourceCode: ${{ parameters.backupSourceCode}} # can be changed to hardcoded value true or false
        backupVariableGroups: ${{ parameters.backupVariableGroups}} # can be changed to hardcoded value true or false
        repositories: # List of all repositories that you want to backup.
          - self # Self means current repo where this file will be located. The current repo will be backed up by default.
          - "git://ProjectName1/RepoName1" # CHANGE - Please use syntax "git://ProjectName/RepoName" for all other projects/repos. If Project name and/or Repo name contains spaces, then use it as it is without escaping.
          - "git://ProjectName1/RepoName2" # CHANGE
          - "git://ProjectName2/RepoName1" # CHANGE
          - "git://ProjectName2/RepoName2" # CHANGE
          - "git://ProjectName3/RepoName1" # CHANGE
          - "git://ProjectName3/RepoName2" # CHANGE
        serviceConnectionName: 'AzDevOps-Bkp-Svc' # CHANGE - name of the service connection in Azure DevOps for the Azure subscription
        storageAccountName: 'devopsstorageaccount'  # CHANGE - name of the target Azure storage account. MUST EXISTS before execution!
        backupContainerName: 'azure-devops-container' # CHANGE - name of the container in the storage account. MUST EXISTS before execution!
        blobDirectoryName: 'az-devops-backup' # CHANGE - name of the blob directory where you wish to store the backup
        # Other optional parameters:
        #backupArtifacts: true/false - enable backup of artifact, default is false
        #feedPath: path to artifact's feed
        #packageName: package name
        #packageVersion: pacakge version, usually from naming template

The second YAML definition that you need, is the backup template itself which contains all the logic to back up the source code, variable groups (optionally), and/or artifacts (optionally). You don’t need to update any parameters in this file, all the parameters are defined in the above YAML file.

You can get a copy of this YAML file directly from my GitHub.

parameters:
  - name: serviceConnectionName
    displayName: 'Service Connection Name'  
  - name: storageAccountName
    displayName: 'Storage account Name'    
  - name: backupContainerName
    displayName: 'Container name in storage account'    
  - name: blobDirectoryName
    displayName: 'Blob Directory Name' 
  - name: envName
    displayName: 'Environment'
    type: string
    default: 'DevOps Backup'

  - name: backupSourceCode
    displayName: 'Backup Azure DevOps Source Code'
    type: boolean
    default: true
  - name: repositories
    displayName: Repositories to backup (use "git://ProjectName/RepoName" for other repos)
    type: object
    default:
       - self
       #- "git://ProjectName/RepoName"
  
  - name: backupVariableGroups
    displayName: 'Backup Azure DevOps Variable Groups'
    type: boolean
    default: true
  - name: devOpsOrgUrl
    displayName: 'DevOps Organization URL'
    type: string
    default: ' '

  - name: backupArtifacts
    displayName: 'Backup Azure DevOps Artifacts'
    type: boolean
    default: false
  - name: feedPath
    displayName: 'Feed path'
    type: string
    default: ' '
  - name: packageName
    displayName: 'Package name in feed'
    type: string
    default: ' '
  - name: packageVersion
    displayName: 'Package version (usually from naming)'
    type: string
    default: ' '

jobs:
  - ${{ if eq(parameters.backupVariableGroups, true) }}:
    - deployment: backupVariableGroup
      variables:
      - name: backupDateTime
        value: $[format('{0:yyyy}-{0:MM}-{0:dd}T{0:HH}-{0:mm}-{0:ss}', pipeline.startTime)]
      - ${{ if eq(parameters.devOpsOrgUrl, ' ') }}:
        - name: actualDevOpsUrl
          value: ${{ variables['System.TeamFoundationCollectionUri'] }}
      displayName: Backup Variable Group
      environment: ${{ parameters.envName }}
      strategy:
        runOnce:
          deploy:
            steps:
              # Getting access token for the Repo
              - checkout: self
                persistCredentials: true
                clean: true
              # You don't need to use az devops login in case of using AZURE_DEVOPS_EXT_PAT env variable
              # Proof: https://learn.microsoft.com/en-us/azure/devops/cli/log-in-via-pat?view=azure-devops&tabs=windows
              - script: az devops configure --defaults organization=$(actualDevOpsUrl) project="$(System.TeamProject)" 
                displayName: 'Set default Azure DevOps organization and project'
                env:
                  AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)

              - script: az pipelines variable-group list
                displayName: 'Test Az pipelines command'
                env:
                  AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
              
              - pwsh: |
                  $backupFolder = New-item (Join-Path "backup" "$(backupDateTime)")  -ItemType Directory -Force
                  $bgFolder = Join-Path $backupFolder.FullName "variable-groups"
                  # Get all variable groups
                  $groups = ConvertFrom-Json "$(az pipelines variable-group list)"
                  # echo $groups
                  $groups | foreach {
                    $groupName = $_.name

                    # Prepend VariableGroups folder name
                    $filePath = Join-Path $bgFolder "$groupName.json"

                    # Save the variable group to a file
                    ConvertTo-Json $_ | New-Item $filePath -Force
                  }
                displayName: 'Save variable groups'
                workingDirectory: $(System.DefaultWorkingDirectory)
                env:
                  AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
              - task: CmdLine@2
                inputs:
                  script: |
                    az storage blob upload-batch -d ${{ parameters.backupContainerName }} --source "$(System.DefaultWorkingDirectory)/backup" --connection-string "${{ parameters.backupStorageAccountConectionString }}" --overwrite
                  workingDirectory: '$(System.DefaultWorkingDirectory)'

  - ${{ if eq(parameters.backupArtifacts, true) }}:
    - deployment: backupArtifacts
      displayName: Backup DevOps Artifacts
      variables:
        - name: backupDateTime
          value: $[format('{0:yyyy}-{0:MM}-{0:dd}T{0:HH}-{0:mm}-{0:ss}', pipeline.startTime)]
      environment: ${{ parameters.envName }}
      strategy:
        runOnce:
          deploy:
            steps:
              #frontendbuild
              - task: DownloadPackage@1
                displayName: Download artifacts
                inputs:
                  packageType: 'upack'
                  feed: ${{parameters.feedPath}} #'$(az._global.projectName)/$(az._global.artifacts.frontendbuild)@local' # stage=local/Release/Prerelease
                  definition: ${{ parameters.packageName }}
                  version: '*'
                  downloadPath: '$(System.DefaultWorkingDirectory)/backup/$(backupDateTime)/Artifacts/${{ parameters.packageName }}/${{parameters.packageVersion}}'

              - script: |
                    az storage blob upload-batch -d ${{ parameters.backupContainerName }} --source "$(System.DefaultWorkingDirectory)/backup" --connection-string "${{ parameters.backupStorageAccountConectionString }}" --overwrite
                workingDirectory: '$(System.DefaultWorkingDirectory)'
                displayName: 'Publish to storage account'

  - ${{ if eq(parameters.backupSourceCode, true) }}:
    - deployment: backupSourceCode
      displayName: Backup Source Code
      variables:
        - name: backupDateTime
          value: $[format('{0:yyyy}-{0:MM}-{0:dd}T{0:HH}-{0:mm}-{0:ss}', pipeline.startTime)]
      environment: ${{ parameters.envName }}
      strategy:
        runOnce:
          deploy:
            steps:
            - ${{ each repoToBackup in parameters.repositories }}:
              - checkout: ${{ repoToBackup }}
                persistCredentials: true
                clean: true
                path: 
            - script: |
                  /bin/mkdir -p backup/$(backupDateTime)/SourceCode
                  ls
                  find . -maxdepth 1 ! \( -name backup -o -name '.' \) -exec cp -r {} backup/$(backupDateTime)/SourceCode/ \;
              workingDirectory: '$(System.DefaultWorkingDirectory)'
              displayName: Prepare backup folder
            - ${{ if le(length(parameters.repositories), 1) }}:
              - script: |
                    zip -r -m "devOpsCodeBackup.zip" .
                    ls -Al                   
                workingDirectory: '$(System.DefaultWorkingDirectory)/backup/$(backupDateTime)/SourceCode'
                displayName: Backup single repo
            - ${{ if gt(length(parameters.repositories), 1) }}:
              - script: |
                    for i in */; do zip -r -m "${i%/}.zip" "$i"; done
                    ls -Al                                      
                workingDirectory: '$(System.DefaultWorkingDirectory)/backup/$(backupDateTime)/SourceCode'
                displayName: Backup all repos
            - task: AzureCLI@2
              displayName: Azure Storage Blob Upload Batch
              inputs:
                scriptType: 'pscore'
                scriptLocation: 'inlineScript'         
                azureSubscription: ${{ parameters.serviceConnectionName }}              
                inlineScript: | 
                    az storage blob upload-batch -d ${{ parameters.backupContainerName }} --account-name ${{ parameters.storageAccountName }} --destination-path ${{ parameters.blobDirectoryName }} --source "$(System.DefaultWorkingDirectory)/backup" --overwrite

Now we have both files saved in our main (self) repository as shown in the figure below:

Azure DevOps multiple repos backup .yml
Azure DevOps multiple repos backup .yml

In the next step, you need to create a new pipeline and select the Git repository where you saved the first YAML file. In this example, the file is called “backup-all-repos.yml“. Then click Continue.

Select an existing Azure Pipeline YAML file
Select an existing Azure Pipeline YAML file

Next, click Run and test the pipeline. When you run the pipeline for the first time, you need to permit it to access before the first run can continue. Please note that this is only needed once. Select the waiting “Backup Source Code” job, then click View and Permit twice to confirm. Please note that granting permission will permit the use of Environment ‘DevOps Backup’ for all waiting and future runs of this pipeline.

Effective Way To Backup Azure DevOps Repositories 17

Last, you can configure and create a schedule on when you want to run the pipeline and trigger the backup, then click Save.

Configure schedules for pipelines to trigger backup
Configure schedules for pipelines to trigger backup

If you run the pipeline more than once per day or manually, then the backup will be created based on the date and time when you run the pipeline. The backup will be uploaded to a separate blob directory appended by the date and time as shown in the figure below.

Azure DevOps Backup Multiple Repos
Azure DevOps Backup Multiple Repos

There you have it. Happy Azure DevOps Backup!

Summary

In this article, we illustrated all the steps that you need to back up a single Azure DevOps repository, as well as how to back up multiple repositories in a single Azure DevOps project, and optionally variable groups and artifacts backup.

The logic of the YAML Pipeline is, that if there’s any change to your source repository, the pipeline will be triggered and it takes the latest copy in the System.DefaultWorkingDirectory (CmdLine task) and archive this copy into a Backup.zip file, then the Azure File copy task will copy the .zip file to Azure blob storage.

You can also copy it to Azure VM which allows you to create a daily/weekly backup for your VM, please check the Azure File Copy task for more information.

With the help of simple tasks, we can automate and produce a full copy of the repository that could be easily restored or imported into the new Azure DevOps project.

__
Thank you for reading my blog.

If you have any questions or feedback, please leave a comment.

-Charbel Nemnom-

Photo of author
About the Author
Charbel Nemnom
Charbel Nemnom is a Senior Cloud Architect with 21+ years of IT experience. As a Swiss Certified Information Security Manager (ISM), CCSP, CISM, Microsoft MVP, and MCT, he excels in optimizing mission-critical enterprise systems. His extensive practical knowledge spans complex system design, network architecture, business continuity, and cloud security, establishing him as an authoritative and trustworthy expert in the field. Charbel frequently writes about Cloud, Cybersecurity, and IT Certifications.
Previous

What’s New in System Center 2019 Data Protection Manager Update Rollup 2

How to Lower Azure File Shares Cost with Hot and Cool Tiers

Next

36 thoughts on “Effective Way To Backup Azure DevOps Repositories”

Leave a comment...

  1. Do you have these YAML files available for download? The formatting gets lost with a copy and paste. Really like the idea but struggling to get the file formatted correctly.

  2. Hello David, thanks for the comment!
    Yes, you can download these YAML files. I have updated the article to include the download URL for each one.
    Hope it helps!

  3. It is a great help. Backing up multiple repositories worked for me.

    But I can’t restore it to new repo by following command:

    .git > git push –mirror https://dev.azure.com/organization-name/project-name/_git/new-repo-name

    Enumerating objects: 45, done.
    Counting objects: 100% (45/45), done.
    Delta compression using up to 8 threads
    Compressing objects: 100% (44/44), done.
    Writing objects: 100% (45/45), 13.68 KiB | 2.28 MiB/s, done.
    Total 45 (delta 10), reused 0 (delta 0), pack-reused 0
    remote: Analyzing objects… (45/45) (6 ms)
    remote: Validating commits… (15/15) done (0 ms)
    remote: Storing packfile… done (47 ms)
    remote: Storing index… done (47 ms)
    To https://dev.azure.com/organization-name/project-name/_git/new-repo-name
    – [deleted] main
    * [new reference] origin/ivan -> origin/ivan
    * [new reference] origin/main -> origin/main
    * [new reference] origin/master -> origin/master

    I cannot find any files in the new repository on Azure DevOps. Can you do me a favor? Thank you!

  4. Hi,

    is it possible to use a single project to backup multiple projects repos?

    I know you can clone multiple repos from the same project but I could really do with a single project for backup which then clones all repos in all projects?

  5. Hello Ivan, thanks for the feedback and comment!
    I am happy to hear that backing up multiple repositories is working for you.
    Now in regards to restoring to a new repo. I just did it on my side by creating a new Git repository without initializing it and then I restored my backup using the following commands in sequence:
    Make sure NOT to browse to the .git folder when restoring, just go to the extracted folder of the .zip backup.

    git checkout master
    git remote remove origin
    git remote add origin https://dev.azure.com/organization-name/project-name/_git/new-repo-name
    git push -u origin --all

    It’s working for me without any issue. I can find the restored files in the new repository on Azure DevOps including all the history.
    Let me know if it works for you.

  6. Hello Paul, thanks for the comment and the great question!
    Yes, this is possible. What you need to do, is to define all repos in all projects that you need to backup manually.
    In the “backup-all-repos.yml” file, you need to list all project names and repos that you want to backup as follows:

    "# CHANGE - Please use syntax "git://ProjectName/RepoName" for all other projects/repos. If Project name and/or Repo name contains spaces, then use it as it is without escaping.
    "git://ProjectName1/RepoName1"
    "git://ProjectName1/RepoName2"
    "git://ProjectName2/RepoName1"
    "git://ProjectName2/RepoName2"
    "git://ProjectName3/RepoName1"
    "git://ProjectName3/RepoName2"

    You can backup any repo from any other project.
    Hope it helps!

  7. Hi again and thanks for your help, I also had to add the build service for the backup project to the project I wanted to backup repos from (because we have an org settings of “Limit job authorization scope to current project for non-release pipelines”).

    Now we have some projects with hundreds of repos, is there a way to wildcard this?

  8. Hello Paul, thanks for the follow-up!
    Unfortunately, with the current version that I have, we need to list/add all repositories “name” manually.
    I will see if I can loop through all the repositories in an project and release a new update.
    Thanks for the feedback!

  9. Hello,

    Great article! I’m trying to calculate how much will it cost to implement this solution. If I understood correctly I will be using 1 Microsoft-hosted CI/CD Pipeline for this implementation. I can use 1 for my 20+ repositories following the above instructions?

    Thanks,
    Gladys

  10. Hello Gladys, thanks for the comment and feedback!
    Yes, you’ve got it right. You can efficiently utilize one Microsoft-hosted CI/CD Pipeline for your 20+ repositories by following the instructions provided in this article.
    If you have any more questions or need further clarification, feel free to ask.
    Hope it helps!

  11. I am seeing an issue when backing up multiple repos for some reason I cannot find yet, the command is adding a trailing ‘ on the end of the URL so it fails to connect.

    In the error message below the first ‘ before https is not being included in the URL yet the trailing one is.

    remote: TF401019: The Git repository with name or identifier Demo-Server does not exist or you do not have permissions for the operation you are attempting.
    fatal: repository ‘https://dev.azure.com/demo/Demo/_git/Demo-Server/’ not found
    ##[warning]Git fetch failed with exit code 128, back off 3.913 seconds before retry.

  12. Hello Christian, thanks for the comment and for reporting this error!
    That’s strange because I have it working in my environment without any issue. I am backing up multiple repos.
    I believe that you entered the Git URL wrong for the repo that you want to protect.
    Could you please double check the “backup-all-repos.yml” file?
    Please use syntax “git://ProjectName/RepoName” for all other projects/repos. If Project name and/or Repo name contains spaces, then use it as it is without escaping.
    Did you get the copy of the YAML files from the GitHub link that I shared above?
    Hope it helps!

  13. Hello, Charbel, thanks for this great article. It sounds like exactly what we need, except for one thing. We are a very small company, and my manager is concerned about the cost of the azure storage. I’m trying to figure out if there is a way to have this save to our own server instead of to blob storage, but I’m not sure how to go about it. Do you have time for any suggestions?

  14. Hello Rich, thanks for the feedback and the great question!
    I understand your concern about the cost of Azure Storage.
    As a side note, if you know the size of the repository that you want to backup, you can calculate the price of Azure Storage and use the cheapest tier available like Cool, Cold, or Archive to bring the cost down (Check this article for more information).
    While Azure DevOps offers native integration with Azure Storage for repository backups, there are alternative approaches you can consider to save backups to your own server.
    First, you need to set up a self-hosted agent in your environment. This agent can run on your own server. This server could be running on-premises or in any other cloud (Learn about self-hosted agents).
    Then, you configure the backup process on this self-hosted agent, ensuring that the backups are stored locally on your server. The backup will be saved locally on your self-hosted agent server under the following path: ‘$(System.DefaultWorkingDirectory)/backup’.
    For a step-by-step guide, check the following article on how to create and configure a Self-Hosted Agent for Azure Pipelines.
    Last, make sure to remove and delete the ‘AzureCLI@2’ task for Azure Storage Blob Upload Batch in the YAML example above.
    Hope it helps!

Let us know what you think, or ask a question...