An Attempt to initialize VM saved state failed – Could Not Create or Access Saved State File #HyperV #SCVMM

In this article, we will share with you how to resolve the attempt to initialize VM saved state failed and could not create or access the saved state file.

Introduction

Hello Folks,

After a few weeks away from work on vacation, I’ve finally managed to sit back and go through all my 30 Hyper-V hosts across different remote sites from a single pane of glass, of course, you guessed it? yes, yes… Virtual Machine Manager.

What I’ve come across from browsing through the console one Virtual Machine in a particular remote site is in a Stopped state.

And of course, what is the first thing you want to try to do is to power on that VM!

The issue

The unexpected behavior, the VM failed to start with the following error:

Error (12700)
VMM cannot complete the host operation on the hv01.dc.net server because of the error: ‘DC01’ could not initialize. (Virtual machine ID 7431245E-0FB3-45D3-B2E2-014124480F42)

‘DC01’ could not create or access saved state file D:\VirtualMachines\DC01\Virtual Machines\7431245E-0FB3-45D3-B2E2-014124480F42\7431245E-0FB3-45D3-B2E2-014124480F42.vsv. (Virtual machine ID 7431245E-0FB3-45D3-B2E2-014124480F42)
Unknown error (0x8001)

Recommended Action
Resolve the host issue and then try the operation again.

As we can see in the error above that the Virtual Machine could not access the saved state file .vsv.

The next logical thing is to remote into the Hyper-V host and try to dive deeper.

Again the VM failed to start.

As you can see in the above self-explanatory error that you do not have permission to perform this operation, therefore the issue is security permission.

Two errors are logged in the Event Viewer as well:

Event ID 3040

Event ID 3080

Back to basics, what files are used to create a virtual machine in Hyper-V:

• .XML files
  • These files contain the virtual machine configuration details. There is one of these for each virtual machine and each checkpoint of a virtual machine. They are always named with the GUID used to identify the virtual machine internally
• .BIN files
  • This file contains the memory of a virtual machine or checkpoint that is in a saved state.
• .VSV files
  • This file contains the saved state from the devices associated with the virtual machine.
• .VHD/VHDX files
  • These are the virtual hard disk files for the virtual machine.
• .AVHD files
  • These are the differencing disk files used for virtual machine checkpoints aka (snapshots).
• .VFD files
  • These are for virtual floppy disk files and rarely do you use them.

What are the symptoms that we observed so far:

• Powering on the VM in Virtual Machine Manager, Hyper-V Manager, or PowerShell would result in an error – could not create or access the saved state file.
• Error 3040 and 3080 were logged into the Hyper-V Worker event log.

I will browse into the .vsv and .bin files of the affected virtual machine and check its security permission.

.BIN

.VSV

If we compare the .vsv and .bin files for a running VM on the same host, we notice something different:

The GUID name of the virtual machine has full control over those files, but not on the affected VM.

One important point is that the GUID is the security context used to access the various files that make up the VM.  The VM Worker Process (vmwp.exe) will leverage this to work with the files. You can open up task manager or fire up PowerShell and see the VM Worker Process details, the GUID is listed in the user name field:

Let’s grant permissions to the .bin and .vsv files, you can use ICacls.exe from the command prompt (cmd):

The syntax is the following:

D:\VirtualMachines\DC01\Virtual Machine\7431245E-0FB3-45D3-B2E2-014124480F42\ICacls.exe 7431245E-0FB3-45D3-B2E2-014124480F42.bin /grant "NT VIRTUAL MACHINE\7431245E-0FB3-45D3-B2E2-014124480F42":(F)

D:\VirtualMachines\DC01\Virtual Machine\7431245E-0FB3-45D3-B2E2-014124480F42\ICacls.exe 7431245E-0FB3-45D3-B2E2-014124480F42.vsv /grant "NT VIRTUAL MACHINE\7431245E-0FB3-45D3-B2E2-014124480F42":(F)

You need to adjust and match your GUID including the VM storage location.

Once I set the NTFS security permissions, I tried to power on the VM again!

And…

Ehhhhh! What security permission is still missing

Let’s browse to the root folder of the affected Virtual Machine and compare the NTFS security permission with another running VM.

To illustrate this, you can fire up PowerShell and compare side by side using my favorite compare-object cmdlet:

PS C:\>$acl = Get-Acl -Path "D:\VirtualMachines\DC01"
PS C:\>$acl.access > C:\DC01.txt
PS C:\>$acl = Get-Acl -Path "D:\VirtualMachines\DHCP01"
PS C:\>$acl.access > C:\DHCP01.txt
PS C:\>Compare-object -referenceobject $(get-content C:\DHCP01.txt) -differenceobject $(get-content C:\DC01.txt)

What you can see in the SideIndicator outputs the NT Virtual Machines\Virtual Machines IdentityReference is available on the left arrow which means the reference running VM ‘DHCP01’ and not available on the affected VM ‘DC01’.

If you look into NT Virtual Machines\Virtual Machines NTFS advanced security permission using the UI, you can see below:

Let’s grant the missing security permissions to the root DC01 Virtual Machine folder from ‘DHCP01’ using PowerShell.

PS C:\>$acl = Get-Acl -Path "D:\VirtualMachines\DHCP01"
PS C:\>Set-Acl -Path "D:\VirtualMachines\DC01" -AclObject $acl
PS C:\Start-VM DC01
PS C:\Get-VM DC01

And here you go

Conclusion

The best course of action is to never touch the security permission for the Virtual Machine files once they are working, and don’t change inheritance or modify any of these automatically added items, because when you first build a virtual machine, Hyper-V will set the security permission appropriately.

Hope this helps.

Enjoy your day!

Cheers,
/Charbel