Advanced Hyper-V Replica Monitoring via #PowerShell #WS12R2 #HyperV #SAPIEN

Get-HyperVReplicaHealthV1.0-00

Hyper-V Replica can be your lifesaver if a disaster occurs, and at the same time it can be your enemy if you are not monitoring it.

If you have used Hyper-V Replica (of course you did Winking smile), then you will know about the replica status column in Hyper-V Manager console. This is the part of the UI where you can see the state of replica is: Normal, Warning or Critical. A simple interpretation of these states is the following:

  • Normal: No worries, everything is fine! A virtual machine with Normal state if any of the following happen:
    • Less than 20% of replication attempts have been missed.
    • The average latency. That’s important because Hyper-V Replica wants to replicate the Hyper-V Replica Log File (HRL) every 30 sec, 5 min or 15 min. This depends on the replication frequency set.
  • Warning: There have been some problems, but Hyper-V will try to fix them automatically for you. A virtual machine has a warning state if any of the following happen:
    • The initial replica copy or synchronization is not completed.
    • More than 20% of replication cycles have been missed during the current monitoring interval.
    • One hour or more has passed since the last send replica was successfully sent/received.
    • The source virtual machine’s replication is in paused state.
    • Failover has been started, but the reverse replication has not been started.
  • Critical: Sorry, things have gone past the point where Hyper-V can fix them for you, and therefore you need to intervene. A virtual machine has a critical state if any of the following happen:
    • The source primary host is unable to send Hyper-V traffic to the replica host, this could be network or firewall issues.
    • The replication is suspended on the primary or on the replica host.

One of the problem is that things can go really bad if you have Hyper-V replica in a critical state for a long period of time, the (AVHD/AVHDX) differencing disk will grow and can make a big mess on the primary server, and the next thing, you want to know when things go wrong right? because if there is something going wrong in the environment that causes the virtual machines to go into Warning or Critical state – I want to know! And I want to fix it so it does not happen again! I do not want to think that life is good when actually I have a problem to fix Winking smile

As a side note: Hyper-V Replica creates an AVHDX to handle the initial replication of the virtual machine (i.e. getting the first copy of the virtual machine across the replica host). Hyper-V Replica will merge this disk out once it has successfully made a full copy of the virtual machine, virtual hard disk (which can take a while).

So for this reason, I have created a nifty PowerShell script which e-mail the Hyper-V administrator when something goes wrong with Hyper-V Replica in a nice dashboard HTML format.

This script will check first Hyper-V Replica TCP listener on Port 80, because Hyper-V Replica requires HTTP and HTTPS rules to be enable in Windows Firewall, and then e-mail the Replication State, Replication Mode, the Target frequency of the replica (30 sec, 5 min or 15 min), and how far the replica virtual machine is behind from the primary virtual machine (Delta in min), then the last replication time and the missed replica count used by Virtual Machines which are in Warning and Critical states only in an HTML tabular format to the specified email address. This of course can be used along with a task scheduler to get a notification when something deviating from the normal on a regular basis for monitoring purposes (Please refer to the end of this article to automate the task).

I want to mention that this tool was created using SAPIEN PowerShell Studio 2015 which is a great support when you are creating advanced scripts or GUI Tool Making like the following tool Winking smile.

Get-HyperVReplicaHealthV1.0-04

How to use:

  • Use –PrimHyperV01 parameter, if you want to change the primary Hyper-V host and get info for all primary virtual machines. If not, it will use the Hyper-V host which is defined by default in the script.
  • Use –RepHyperV02 parameter, if you want to change the replica Hyper-V host and get info for all primary virtual machines. If not, it will use the Hyper-V host which is defined by default in the script.
  • Use –ExtHyperV03 parameter, if you want to check the extended Hyper-V Replica host and get info for all primary virtual machines. If not, it will use only the primary and replica Hyper-V hosts.
  • You can run the script on a remote management machine loaded with Hyper-V PowerShell module which is the preferred way, or run it directly on the Hyper-V Host which I don’t recommend.

Examples:

Get-HyperVReplicaHealthV1.0-05

This example will check first the HTTP connectivity to the Primary and Replica Hyper-V server, If both nodes are reachable, then it will look for primary Virtual Machines that are in Warning and Critical state and send you an alert.

Get-HyperVReplicaHealthV1.0-06

This example will check first the HTTP connectivity to the Primary, Replica and Extended Replica Hyper-V Hosts, If all nodes are reachable, then it will look for primary Virtual Machines that are in Warning and Critical state and send you an alert.

Sample Output:

Node is not Reachable Notification!

Get-HyperVReplicaHealthV1.0-01

Warning State Notification!

Get-HyperVReplicaHealthV1.0-02

Critical State Notification!

Get-HyperVReplicaHealthV1.0-03

Last but not least, you can automate the invocation of the Get-HyperVReplicaHealth script using Task Scheduler to run every hour and send you an alert if something goes wrong in Hyper-V Replica. To do this, you can use the Register-ScheduleJob cmdlet. Here’s an example to monitor your Hyper-V Replica Infrastructure every one hour:

You can download a copy of this script from TechNet Gallery here.

This is version 1.0, do you have other scenarios? Please leave your feedback below.

In version 2.0, I will include Hyper-V Replica Broker in a Cluster scenario.

Until then… Enjoy your day!

Cheers,
/Charbel

Posted in Hyper-V, PowerShell Tagged with: , , , ,

#Microsoft #MVP Virtual Conference #MVPbuzz

MVP15_MicrosoftMVP_VC_WebBanner

Register to attend the Microsoft MVP Virtual Conference!

Hello folks, I wanted to let you know about a great free event that Microsoft and the MVPs are putting on, May 14th & 15th.  Join Microsoft MVPs from the Americas’ region as they share their knowledge and real-world expertise during a free event, the MVP Virtual Conference.

The MVP Virtual Conference will showcase 95 sessions of content for IT Pros, Developers and Consumer experts designed to help you navigate life in a mobile-first, cloud-first world.  Microsoft’s Corporate Vice President of Developer Platform, Steve Guggenheimer, will be on hand to deliver the opening Key Note Address.

Why attend MVP V-Conf? The conference will have 5 tracks, IT Pro English, Dev English, Consumer English, Portuguese mixed sessions & Spanish mixed sessions, there is something for everyone! Learn from the best and brightest MVPs in the tech world today and develop some great skills!

Be sure to register quickly to hold your spot and tell your friends & colleagues. Register now and feel the power of community!

clip_image001

Enjoy the virtual conference!

Cheers,
/Charbel

Posted in Hyper-V, MVP Tagged with: , , ,

The Available Processor Sets of the Underlying Physical NICs Belonging to the LBFO Team NIC Are not Configured Correctly #HyperV #VMQ #RSS #LBFO

Hello folks,

I have recently upgraded my Hyper-V host and my network infrastructure to 10Gb.

As soon as I moved the Hyper-V virtual switch to the LBFO team with 2X10Gb, I start receiving the following error:

LBFO-ID106-01

If we look deeper in the event log, we can see the reason is self-explanatory.

LBFO-ID106-02

So what is sum-queue mode and what is min-queue mode?

A while ago, I posted a detailed article on How To Enable and Configure VMQ/dVMQ on Windows Server 2012 R2 with below Ten Gig Network Adapters.

Please make sure to check the article before you proceed with the resolution.

As a quick recap, the SUM-Queues mode is the total number of VMQs of all the physical NICs that are participating in the team, however the MIN-Queues mode is the minimum number of VMQs of all the physical NICs that are participating in the team.

The question is why we don’t get the same error when you have 1Gb network adapters? because when using 1Gb network adapters VMQ is disabled by default, because Microsoft don’t see any performance benefit to VMQ on 1Gb NICs, and a single core can keep up at ~3.5Gb throughput without any problem.

If you need to enable VMQ on 1Gb NICs, please refer to this article.

In my scenario, I am using 2X10Gb adapters configured with Switch independent teaming mode and Dynamic as distribution mode.

Distribution mode→Teaming mode↓ Address Hash modes Hyper-V Port Dynamic
Switch independent Min-Queues Sum-of-Queues Sum-of-Queues
Switch dependent Min-Queues Min-Queues Min-Queues

If you look in the table above, you can see that I am using the Sum-queue mode.

First, we need to check the total number of VMQs for my LBFO team.

LBFO-ID106-03

As you can see VMQ is enabled True, but the Base and Max processors for both 10Gb adapters are set to 0 and Max to 16, therefore the processor sets is overlapping, because the LBFO team is set up for Sum of Queues, the network adapters in the team have to use non-overlapping processor sets.

I have here one Converged Virtual Switch with 2X10Gb NICs teamed and 63 Queues for each NIC used for vNICs on the host and for vmNICs for VMs, so the total number of VMQs for the LBFO team is 126 Winking smile.

You may wonder why 63 queues and not 64, in my scenario here is 128 (64 NIC1 + 64 NIC2), but 1 VMQ queue is reserved by the system so you’ll see 63 per port and 126 per LBFO team.

Before we start configuring the VMQ for each adapter, we need to determine if Hyper-threading is enabled in the system by running the following cmdlet:

LBFO-ID106-04

As you can see we have the NumberOfLogicalProcessors as twice as the NumberOfCores, so my server has two 8-core CPUs and HT is enabled, we can see 32 LPs in task manager.

LBFO-ID106-05

Let’s start configuring virtual machine queue for both adapters.

Since my team is in Sum-Queues mode, the team members’ processors should be, non-overlapping or with little overlap as possible. For example, in my scenario I have a 16-core host (32 logical processors) with a team of 2X10Gbps NICs, I will set the first NIC1 to use base processor of 0 and use max processors 8 cores (so this NIC would use processor 0, 2, 4, 6, 8,10,12,14 for VMQ); the second NIC2 would be set to use base processor 16 and use 8 cores as well (so this NIC would use processor 16, 18, 20, 22, 24, 26, 28, 30 for VMQ).

As best practice, please make sure the base processor is not set to 0, because the first core, logical 0, is reserved for default (non-RSS and non-DVMQ) network processing. 

Let’s open PowerShell and Set-NetAdapterVmq accordingly for each NIC:

LBFO-ID106-06

Let’s verify now that VMQ is applied:

LBFO-ID106-07

As you can see now, the baseVmqProcessor for NIC1 is 0 and the baseVmqProcessor for NIC2 is 16.

So what we have done in this case, the 126 queues are spread across the 16 processors, the first NIC in my example has 63 queues, so it can spread anywhere from processor 0 to 15, and the second NIC from processor 16 to processor 31. Keep in mind that all 16 CPUs will be used since I have more queues than CPUs. However, if you have for example 8 queues per NIC then no more than 8 CPUs will be used since there are only 8 queues.

But… after I set the VMQ, the error did not go away Sad smile.

LBFO-ID106-09

Why?

Because as I mentioned in the beginning of this article, I am using one Converged Team for vmNIC (VMs) and for vNICs in the host as well.

If we look at the RSS on host, we can see the Base and Max processors for NIC1 is set to 0 and NIC2 is set to 16 as well, therefore the processor sets is overlapping with VMQ.

As a side note and best practice, you should split the vNICs on the Host from the vmNICs on a two separate physical adapters (teamed). 

LBFO-ID106-08

In this case, we will roughly split between RSS and Dynamic VMQ 50/50.

The 16 logical processors on CPU0 (0–15) will be used by RSS. The remaining 16 logical processors of CPU1 (16–31) will be used by DVMQ.
The settings for the two 10Gb NICs will depend again on whether NIC teaming is in Sum-of-Queues mode or in Min-Queues mode. NIC1 (Fiber01) and NIC2 (Fiber02) are in a Switch-Independent and Dynamic mode team, so they are in Sum-of-Queues mode. This means the NICs in the team need to use non-overlapping processor sets. The settings for the two 10Gb NICs will therefore be illustrated as the following:

LBFO-ID106-10

Note: As per Microsoft that as soon as you bond the Hyper-V Virtual Switch to the LBFO team, the RSS will be disabled on the host and VMQ will be enabled, in other words the Set-NetAdapterRss actually does not have effect here and the Set-NetAdapterVmq will take precedence, therefore if we look again, we can see that RSS will align with VMQ.

LBFO-ID106-12

Next, you need to reboot your Virtual Machines in order for the new settings to take effect, because each vmNIC will be assigned one queue once the VM is booted.

Last but not least, you can verify this by running Get-NetAdapterVmqQueue and this will show you all the queues they are assigned across the vmNICs for all VMs on that particular Hyper-V host.

LBFO-ID106-11

Finally, after setting the VMQ and RSS correctly on the system, the error is disappeared Open-mouthed smile 

Hope this helps.

Enjoy your day!
/Charbel

Posted in Hyper-V, Network Tagged with: , , , ,

#HyperV vNext DEMO04: Distributed Storage QoS

Hello folks,

In today’s demo, I will show you a very interesting new feature introduced in Windows Server Technical Preview called Distributed Storage QoS.

If you recall, in Windows Server 2012 R2, Microsoft added Storage QoS feature support for Virtual machine, where you can go in and specify a minimum or maximum cap on a virtual disk and stop it from chewing up all your IOPs on the system.

 HV-vNext-DistStorageQoS-02

• The Minimum IOPS per VHD/X: is not enforced, and is not guaranteed, it’s only informational!
• The Maximum IOPS per VHD/X: to cap the max storage performance.

That’s was great! but it was not enough, Microsoft are investing more and more in this area to brings more functionality.

There a couple of big challenges with Storage QoS today in Windows Server 2012 R2.

The first one is, ok great we got this technology that allows us to go in and say you know for this virtual machine cap it this amount of IOPs, and for the other VM cap it with this amount, and that all works great on a single Hyper-V server!

But to be honest, no one is deploying a Standalone Hyper-V host in production, of course we are leveraging Hyper-V cluster for High Availability.

Storage QoS today it doesn’t really works great if you have a dozen of Hyper-V servers talking to a shared storage at the back-end, because in Windows Server 2012 R2 those Hyper-V servers are not aware that there are competing with each other for storage bandwidth.

Thankfully, in the next release of Windows Server, Microsoft introduced Distributed Storage QoS policy manager directly attached on the Scale Out File Server as a cluster resource.

HV-vNext-DistStorageQoS-01

Of course we still have what we had in Windows Server 2012 R2 where you can go to Hyper-V and go to the virtual machine settings and configure Storage QoS properties on each virtual hard disk, but in the technical preview we can actually now go to the scale out file server cluster and configure the Storage QoS policies there, and this enables a couple of really interesting scenarios:

1- The first scenario, if you have a multiple of Hyper-V Servers talking to the the same file server at the back-end all your storage QoS policies get respected.

2- The second scenario is actually allowing us to do some really cool things where we can now start pulling Storage QoS policies and having a single policy that applies to multiple virtual hard drives/virtual machines instead of just one VM or one virtual hard disk.

Note: One important point to mention that distributed storage QoS works for SOFS deployments only. We do not get this new feature with traditional storage such as SAN or NAS deployments.

So without further ado, let’s switch across my demo system and show you how Distributed Storage QoS works in action Open-mouthed smile

I hope you enjoyed the demo, and I would like to thank you for viewing.

Cheers,
/Charbel

Posted in Hyper-V, vNext Tagged with: , , , , ,

#Microsoft Unveils New #HyperV #Containers Technology and Nano Server For The Next #WindowsServer #CloudOS

Hello folks,

Hyper-V unplugged at number one is yet another amazing milestone achievement for Microsoft.

Microsoft just announced a new Hyper-V Containerization technology as well as new Windows Server deployment called Nano Server, a purpose-built Operating System designed to run born-in-the-cloud applications and containers.

HV-Containers01

Hyper-V Containers and Windows Server Containers [Image Credit: Microsoft]

1- Nano Server is a minimal footprint installation of Windows Server that is highly optimized for the cloud, and ideal for containers.

2- Hyper-V containers offer an additional deployment option with enhanced isolation powered by Hyper-V between Windows Server containers and the Hyper-V virtual machine.

More details about these technologies can be found here and here.

Stay tuned for more details on these new innovations.

Enjoy!

Cheers,
/Charbel

Posted in Containers, Hyper-V Tagged with: , , ,

How to Upgrade #HP #StoreVirtual #VSA From V11.0 To V12.0 On #HyperV 2012 R2 Cluster? #SCVMM #VMM

HPVSAV12-Upgrade01

Hello folks,

Finally! HP released the updated version for Lefthand OS V12.0.
Why? Well there are some features that were announced a quite some time ago that we are waiting for, and which become available now.
LeftHand OS Version 12.0 includes the following enhancements:

Space Reclamation—Reclaim space on thinly and fully provisioned volumes used by Windows Server 2012 or later, and vSphere 5 or later.
StoreVirtual Multi-Path Extension Module (MEM) for vSphere—Provides data path optimization similar to StoreVirtual DSM for Microsoft MPIO.
New REST API for StoreVirtual—Enables automation and scripting of clusters, provisioning and volume management.
StoreVirtual VSA term license management—Enforces term licensing for StoreVirtual VSAs.
Support for Microsoft System Center Virtual Machine Manager 2012 R2. Awesome! Open-mouthed smile
Faster remote copy performance and system reboot times.

A few blogs ago, I showed you how to deploy HP StoreVirtual VSA V11.0 on Hyper-V 2021 R2 Cluster.

You can find the 3 part series below:

Part 1: here
Part 2: here
Part 3:
here 

So make sure to read the previous posts before moving along with the upgrade process.

In today’s blog post, I will show you how to upgrade an existing HP StoreVirtual Infrastructure from V11.0 to V12.0.

HP recommends that you always upgrade to the latest versions of software for the HP StoreVirtual VSA including:
• Firmware
• LeftHand OS
• Centralized Management Console (CMC)
• StoreVirtual VSA
• HP StoreVirtual Command-line Interface (CLI)
• HP StoreVirtual DSM for MPIO
• Application Aware Snapshot Manager

Before we start upgrading the storage virtual Infrastructure, we need to plan first.

Upgrade Plan

1- Verify the StoreVirtual version you are currently running and the upgrade path to the new release.
2- Verify the health of the current storage VSA systems and management groups before you start the upgrade.
3- Validate the health of your Hyper-V Cluster (Test-Cluster).
4- If you are running iSCSI load balancing, verify that a VIP is configured on each cluster.
5- Plan for volume access and the impact the upgrade will have to iSCSI sessions.
6- Schedule maintenance window time for the upgrade.
7- Upgrade the HP StoreVirtual VSA Storage components.
8- Keep and save all log files for HP support if needed.
9- Test the StoreVirtual Infrastructure after the upgrade.

Assuming that you verified your existing StoreVirtual VSA Infrastructure including your Hyper-V cluster and everything is healthy!

HP VSA System:

HPVSAV12-Upgrade04

Hyper-V Cluster 2012 R2:

HPVSAV12-Upgrade03

We will move now to Point 7 according to the plan.

You must upgrade the HP StoreVirtual Storage components in the following order:
Step 1. Upgrade Centralized Management Console (CMC) including the Command-Line Interface (CLI).
Step 2. Upgrade StoreVirtual DSM for Microsoft Windows MPIO.
Step 3. Upgrade LeftHand OS (VSA) and Failover Manager (FOM).
Step 4. Upgrade remaining HP StoreVirtual Application Integration Solution Pack components such as System Center Virtual Machine Manager (SCVMM).

One important note to remember, do not make any configuration changes to the management groups in the Configuration Interface. Otherwise, the upgrade might fail.

To get the updated bits, you have two options:

Option 1. Using the Centralized Management Console (CMC), click Help→Preferences→Upgrades.

HPVSAV12-Upgrade02a

HPVSAV12-Upgrade02

Option 2. Download the updated bits manually from HP StoreVirtual Portal here.

Step1. Upgrading the CMC and the CLI

The CMC and CLI can be upgraded independently of the LeftHand OS software and other HP StoreVirtual Storage components. Be aware that some patches may require an upgrade to the CMC before
installing the patch.

1. Ensure that you have Administrator privileges before starting the upgrade.
2. Launch the CMC installation.
3. Follow the instructions on the installation wizard to complete the upgrade.
HPVSAV12-Upgrade05

HPVSAV12-Upgrade06

HPVSAV12-Upgrade07

The Command-line Interface (CLI) upgrade is straight forward as well, launch the installation and follow the instructions in the wizard.

1. Ensure that you have Administrator privileges before starting the upgrade.
2. Launch the CLI installation.
3. Follow the instructions on the installation wizard to complete the upgrade.

HPVSAV12-Upgrade08

HPVSAV12-Upgrade09

HPVSAV12-Upgrade10

Step 2. Upgrading the StoreVirtual DSM for Microsoft Windows MPIO

1. Ensure that you have Administrator privileges before starting the upgrade.
2. Launch the HP StoreVirtual DSM for Microsoft MPIO Installer.
3. Follow the instructions on the installation wizard to complete the upgrade.
4. Repeat the steps below on each Hyper-V Cluster Node.

HPVSAV12-Upgrade11

Step 3. Upgrading the LeftHand OS and Failover Manager

You can upgrade the storage systems and Failover Managers within a management group or upgrade all the storage systems in the Available Systems pool. When upgrading a management group, the system determines how many storage systems can be upgraded simultaneously while ensuring quorum and volume availability. When upgrading storage systems in the Available Systems list, all storage systems (HP StoreVirtual VSAs and HP StoreVirtual Failover Managers) are upgraded as well.

One important point to mention, if you are running the promotional 1TB license of StoreVirtual VSA or the license is expired, you cannot upgrade to V12 and you are permitted to install urgent patches only.

HPVSAV12-Upgrade12

HPVSAV12-Upgrade13

I am running the 10TB version in my demo so I am entitled to the upgrade Winking smile

1. Ensure that you have Administrator privileges before starting the upgrade.
2. The upgrade must be done through the Centralized Management Console (CMC) only and not through the standard setup.
3. Using the Centralized Management Console (CMC), click on Recommended Patches Available→Upgrades.
4. When you see an upgrade for a management group or for available systems, click Install or Continue next to the item you want to upgrade.

HPVSAV12-Upgrade14

HPVSAV12-Upgrade15

5. Click Install. This is an automated process where every node will be updated 1 at the time so that your volumes remain online (if you have quorum of course – majority of managers).

HPVSAV12-Upgrade16

HPVSAV12-Upgrade17

HPVSAV12-Upgrade18

6. Save the upgrade information to a file, and click Export Information

HPVSAV12-Upgrade19

A quick check proves that my VSA infrastructure is updated to V12.0 Open-mouthed smile.

HPVSAV12-Upgrade20a

HPVSAV12-Upgrade20b

HPVSAV12-Upgrade20c

Let’s now validate the new enhancements in LeftHand OS Version 12.0 for Windows Server 2012 R2 and System Center Virtual Machine Manager 2012 R2:

• Space Reclamation—Reclaim space on thinly and fully provisioned volumes used by Windows Server 2012 or later.
• Integrate HP StoreVirtual VSA v12.0 and SCVMM 2012 R2 using SMI-S

Enable Space Reclamation

Space reclamation is enabled manually at the Management Group level.  Right Click your Management Group and Select Enable Space Reclamation.

HPVSAV12-Upgrade21

Next, you will receive a warning message that once upgraded, you cannot downgrade to previous versions of LeftHand OS that do not support space reclamation.

HPVSAV12-Upgrade24

I suggest checking your Device and RAID status to ensure that everything is OK before Enable Space Reclamation. This is done by selecting your Cluster, followed by the Node and then selecting Storage.

As you can see my RAID Status is Normal.

HPVSAV12-Upgrade22

Enter your Management Group name, in my case HYPERV-MGMT-GROUP1 and accept the disclaimer and Enable Space Reclamation.

HPVSAV12-Upgrade23

Note: You need to make sure that you have at least 3GB of RAM on your VSA systems in order to enable space reclamation, however the recommended amount of memory is 4GB.

HPVSAV12-Upgrade25

After enabling Space Reclamation on the LeftHand OS, please make sure to perform the following steps from the host Hyper-V server:
1. Perform a rescan of the disk devices from the host server to ensure that Space Reclamation is registered with the host server.
2. Run defragmentation on existing volumes from the host OS, to ensure that you are maximizing the available space to be reclaimed.

It is recommended to conduct this process during non-peak production times, as performance may be impacted. The following are the steps to rescan and defragment your CSV volume:

HPVSAV12-Upgrade26

HPVSAV12-Upgrade27

Space Reclamation Test

Space Reclamation provides the capability for the LeftHand OS to request freed space from thinly and fully provisioned volumes and make that space available for new data storage. Space in Virtual Machines may be freed up by deleting files, moving data from one volume to another, and other activities that change the space used in the cluster.

Space reclamation can be performed either on the Hyper-V Cluster Shared Volume after a Storage Migration has taken place or when files have been deleted from within the guest OS.

In this test I’m going to perform a copy of fixed VHDX with 12GB of ISO images to a thinly provisioned Cluster Shared Volume 40GB. Then I will delete the fixed VHDX to force UNMAP (Space Reclamation) to take effect, and watch the space consumed of the Size of the LUN on the VSA, It drops as the data I deleted is in the active part of the LUN. Pretty awesome Open-mouthed smile

HPVSAV12-Upgrade40

After copying the fixed VHDX

  • Volume – 13.45GB consumed space
    • 40GB Thin Provisioned volume with Adaptive Optimization enabled

HPVSAV12-Upgrade41

After deleting the fixed VHDX

  • Volume – 512MB consumed space
    • 40GB Thin Provisioned volume with Adaptive Optimization enabled

HPVSAV12-Upgrade42

Monitoring Space Reclamation

HP have introduced some extra performance statistics to monitor space reclamation which includes:

  • IOPS Space Reclamation
  • Total Latency Space Reclamation (UNMAP)
  • Queue Depth Space Reclamation

These can be accessed by added to the Performance Monitor window so that you can verify the effect of space reclamation on your Store Virtual infrastructure.

HPVSAV12-Upgrade39

Integrate HP StoreVirtual VSA V12.0 and SCVMM 2012 R2 using SMI-S

As I mentioned in the previous post Part 3, System Center Virtual Machine Manager 2012 R2 was not supported with HP VSA LeftHand OS V11.0, however things has been changed now with V12.0.

What are the requirements for the integration?

• Windows Server 2012 R2 Hyper-V hosts.
• SCVMM 2012 R2.
• If you are using Fibre Channel, this only supported using SCVMM Server 2012 R2.

SCVMM storage device discovery and registration

System Center Virtual Machine Manager and Windows PowerShell require an IP address of the storage SMI-S provider to discover, register, and interact with the storage device. Use the Virtual IP Address (VIP) of the cluster to ensure high availability and failover protection.

Open the Fabric workspace in VMM, browse to the Storage, then right click on Providers and select Add Storage Devices.

Select SAN and NAS devices discovered and managed by a SMI-S provider

HPVSAV12-Upgrade31

The cluster virtual IP address of the VSA can be added as storage provider. When adding the Provider IP address make sure you use SSL encryption for the communication, default port (TCP 5989), and protocol SMI-S CIM-XML as showing in below screenshots:

HPVSAV12-Upgrade32

Next, click Import the certificate.

HPVSAV12-Upgrade33

HPVSAV12-Upgrade34

After the provider has been added successfully, all clusters in your StoreVirtual management group will be listed in the Classification and Pools list. New volumes can be provisioned from the available storage pools, used by new virtual machines and presented to Hyper-V hosts that are managed by Virtual Machine Manager via iSCSI.

HPVSAV12-Upgrade35

Click Finish.

HPVSAV12-Upgrade36

And here you go HP VSA Storage Infrastructure completely managed by SCVMM 2012 R2. Fantastic! Open-mouthed smile

HPVSAV12-Upgrade37

SCVMM rapid provisioning provides a method to quickly deploy new virtual machines on top of block storage devices without copying the virtual machines over the network. SCVMM requests a
copy of an existing LUN on the block storage device. The LeftHand OS uses SmartClone volumes for this scenario. SmartClone volumes are fully featured volumes that derive from a snapshot
configured as a clone point of an existing volume.

HPVSAV12-Upgrade38

I hope this in depth article has been informative for you, and I would like to thank you for reading!

Until next time… Enjoy your day!

Cheers,
/Charbel

Posted in Cluster, Hyper-V Tagged with: , , , , , , , , , , ,

#HyperV vNext DEMO03: Production Checkpoints

Hello folks,

In today’s demo I will show you another new feature in Hyper-V vNext called Production Checkpoints.

A little bit of overview about checkpoint (aka snapshot):
A checkpoint is a point-in-time capture of a virtual machine’s storage and memory state, which can then be reverted to at some later point in time.

When Microsoft first designed checkpoints back in Windows Server 2008, they were designed specifically for Dev and Test environment and not for production use, because the virtual machine is unaware of the checkpoint capture and therefore the state could be inconsistent. If you think about it, if you have an exchange server, SQL server, or a file server, when you roll the server back with all the memory state, it thinks that has all the clients are still connected to it, it thinks it still has the IO operations of that you don’t want a server to have at this point in time.

However people are using checkpoints in production Sad smile, so Microsoft in Windows Server Technical Preview introduced Production Checkpoints, which gives you the exact same user experience that you get with checkpoints today, but instead of capturing the memory state, they use VSS (Volume Shadow Copy) infrastructure to get a data consistent snapshot inside the (Guest) Windows virtual machine. However for Linux virtual machine, the file system buffers are flushed out, which makes the OS an application-consistent state and therefore usable in production environment.

WARNING! That this still doesn’t eliminate the need for regular backups. Please keep taking backup!

One important point to mention that you can still use the old-style (standard) checkpoints if you prefer, as well as configure the standard checkpoints to be used if the production checkpoint is not possible (e.g. If the backup integration service isn’t available).

Let’s look together at the Production Checkpoints in action!

DEMO:

I hope you enjoyed the short demo, and I would like to thank you for viewing.

Cheers,
/Charbel

Posted in Hyper-V, vNext Tagged with: , ,

#HyperV vNext DEMO02: Hyper-V Replica Support For Hot Add/Remove of VHDX

Hello folks,

Today I will demo another new big operational feature in Hyper-V Technical Preview, Hyper-V Replica support for Hot Add and Remove of Virtual Hard Disk.

Microsoft added the ability to hot add and remove virtual hard drive to virtual machines that was being replicated. This is fantastic! Why? Because in Windows Server 2012 or 2012 R2, if we add a virtual hard disk to a virtual machine that was being replicated, then the replication would fail, and the replica will go into a critical state. Sad smile

In Windows Server Technical preview that doesn’t happen anymore, when you add a virtual hard drive to a virtual machine it doesn’t get automatically added to the replications set, but the replication will continue to work just fine, and at any stage while the virtual machine is still running and replicated, you can actually go in and use a simple PowerShell cmdlet to add or remove virtual hard drive from the replication disks set, and then Hyper-V Replica will automatically re-synchronize and everything works as expected. Open-mouthed smile

Thank you Microsoft for making Hyper-V Replica experience a lot more seamless.

To add and update the new virtual hard disk to the replica disk set, you will run the following cmdlet:

To remove any virtual hard disk from the replica disk set, you actually use the same PowerShell cmdlet above, but instead of passing all the disks into ReplicatedDisks – you pass in just the disks that you want to replicate, and exclude the ones that you want to remove.

Let’s look together at the Hyper-V Replica enhancement in action!

DEMO:

I hope you enjoyed the demo, and I would like to thank you for viewing.

Cheers,
/Charbel

Posted in Hyper-V, vNext Tagged with: , , ,

#HyperV vNext DEMO01: Hot Add/Remove Memory and Virtual Network Adapters

Hello folks,

In today’s demo, I would like to show you two big new features request in Hyper-V vNext for solving operational problems.

None of us would like to bring the workload offline in order to service the virtual machine.

1- Hot Add and Remove Memory for Static Virtual Machines:

As you know Microsoft introduced Dynamic memory back in Windows Server 2008 R2 SP1.
However we can’t use Dynamic memory on all our workloads like SQL or Exchange Server.
Microsoft is listening to us and they have added hot add and remove memory for Static Virtual Machines.
What is the requirements for static memory?
This is very specifically only if you’re running the technical preview guest on the technical preview Hyper-V host, then you can go in with statically configured memory and change the memory both up and down while the virtual machine is running :)

The interesting part is, If you go to the memory TAB in Hyper-V Manager console even though that this is static memory VM, we can actually see the memory on demand, thus you can size the memory effectively as well. This is fantastic!

2- Hot Add and Remove Virtual Network Adapters:

Hot add and remove virtual network adapters. This is for Generation TWO virtual machines only (WS2012 / W8, WS12R2 / W8.1 or Technical Preview / W10 guests).
You can go in add or remove network adapters while the virtual machine is running!

The other interesting big feature is, the virtual network adapter identification or device naming, this one is actually very interesting Open-mouthed smile.
When you add a virtual network adapter, you can give it a name which is then exposed inside the virtual machine.
This is through PowerShell only at the moment, then from inside the virtual machine you can actually go and check, this adapter is that one, and the other adapter for that one. Very convenient and useful.

Let’s look together at those new features in action!

DEMO:

I hope you enjoyed the demo, and I would like to thank you for viewing.

Cheers,
/Charbel

Posted in Hyper-V, vNext Tagged with: , , ,

#DPM Protection Agent Failed Because The Service Did Not Respond! Internal error code: 0x8099090E #SysCtr

Today is World Of Availability Day!

Everything was running fine for an extended period of time until today Sad smile

The DPM agent reported the following error:

DPM-0x8099090E-01

Error: Data Protection Manager Error ID: 316

          The protection agent operation on APP-V-SQL.LAB.COM failed because the service did not respond.

 

Detailed error code: Internal error code: 0x8099090E

 

Recommended action: If you recently installed a protection agent on APP-V-SQL.LAB.COM, the computer may be restarting. Wait a few minutes after restarting the computer for the protection agent to become available. Otherwise, troubleshoot the problem as follows:

 

1) Check the recent records from the DPMRA source in the Application Event Log on APP-V-SQL.LAB.COM to find out why the agent failed to respond.

2) Verify that the DPM server is remotely accessible from APP-V-SQL.LAB.COM.

3) If a firewall is enabled on the DPM server, verify that it is not blocking requests from APP-V-SQL.LAB.COM.

4) If APP-V-SQL.LAB.COM is a workgroup computer configured to use NETBIOS, ensure that the NETBIOS name of the DPM server is accessible from APP-V-SQL.LAB.COM. Otherwise verify that the DNS name is remotely accessible.

5) If APP-V-SQL.LAB.COM is a workgroup server, ensure that the DPM server has an IPSEC exception to allow communication from workgroup servers.

6) If APP-V-SQL.LAB.COM is a workgroup server the password for the DPM user accounts could have been changed or may have expired on the protected server. To resolve this error, run SetDpmServer with the -UpdatePassword flag on the protected computer and Update-NonDomainServerInfo.ps1 on the DPM server.

7) Restart the DPM Protection Agent service on APP-V-SQL.LAB.COM. If the service fails to start, reinstall the DPM protection agent.

8) If APP-V-SQL.LAB.COM is configured using certificates, ensure that the DPM CPWrapper service is running on DPM Server and APP-V-SQL.LAB.COM. Also ensure that certificates used by both the computers are valid.

The details of the error is self-explanatory, we have several points to troubleshoot.

Let’s start troubleshooting each point.

Point 1: I don’t see any error in the Application Event Log.

Point 2: The DPM server is remotely accessible from the DPM Server and from the Protected Server as well.

Point 3: The Firewall is not blocking any requests.

Point 4: The protected server is in a different domain (Untrusted), the NETBIOS name of the DPM server is accessible from the protected server and vice versa.

Point 5: The protected server is in a different domain (Untrusted), the DPM server can communicate with workgroup servers without any problem.

Point 6: The protected server is in a different domain (Untrusted), the password for the DPM user accounts that is used on the protected server is set to never expire. I tried to reset the password, but unfortunately the agent is still unreachable.

DPM-0x8099090E-02

Point 7: I restarted the DPM Protection Agent service on the protected server, the service started successfully, but unfortunately the agent is still unreachable.

Point 8: No certificates is being used.

None of the suggested solutions solved the issue Sad smile

So what is the issue then?

I will add point 9 to above list and see how can we solve this issue Winking smile

9) If the protected server is a workgroup server, ensure the password for the local user accounts on the DPM Server is set to never expire.

As you noticed, I am protecting domain and non-domain machines in my environment.

The non-domain machines reported this issue only, however the domain machines did not complaint.

When you deploy the DPM agent to Untrusted Servers, you deploy the agent first on the protected server, then you set the DPM Server using SetDpmServer.exe –dpmservername DPM2012 –isnondomainserver –username UNTRUST1, and finally you attach the agent manually on the DPM server using Attach-NonDomainServer.ps1 cmdlet.

Now behind the scene, DPM agent will create a local user account on the protected sever, and then when you attach the agent on the DPM server, the same username will be created locally on the DPM server as well.

Let’s open Computer Management console on the DPM server, under Local Users and Groups, locate the user account and open properties.

DPM-0x8099090E-08

As you can see the User must change the password at next logon Surprised smile

Let’s change it to Password never expires.

DPM-0x8099090E-09

Now refresh the agent again in the DPM console and here you go Open-mouthed smile

DPM-0x8099090E-07

But wait, we didn’t finish yet! Because after 42 days you will experience the same issue again Sad smile

What?… and why after 42 days?

Because if you recall, the local security policy on the server is set by default to expire after 42 days!

DPM-0x8099090E-05

In order to avoid this issue in the future, you need to make sure to set the Maximum password age on the DPM server to Password will not expire (0 days).

DPM-0x8099090E-06

You can add point 9 to your troubleshooting list.

Hope this helps.

Happy World Backup Day!

Cheers,
/Charbel

Posted in Data Protection Manager, System Center, System Center 2012 R2 Tagged with: , , ,

April News: #StarWind Virtual SAN @StarWind #HyperV

Hello folks,

I am glad to share with you the latest news of StarWind for April 2015.

1. Tap Room Meeting: VMware and StarWind Virtual SAN: from Two, to Infinity and Beyond

StarWind_TapRoom
Date: 2 April, 11 am ET / 4 pm BST

Host: Anatoly Vilchinskiy, Head of Support Department, StarWind Software Inc.

Let’s put all cards on the table: VMware is good, their VSAN is good. But does it good for all situations? Get yourself on the most comfortable chair in the Tap Room, open the whisky bottle, and our host will tell you why is it better to start with two shots instead of shugging the whole bottle.

*The only Tap Room, where no one will snatch your cocktail if you leave it on a table.

Virtualize properly!

Book your table here!

2. Free Webinar! Critical SQL Server Databases: Provide HA with SQL Server Failover Clustering and Cluster Shared Volumes

15_Apr_Webinar

Date: 15 April, 11 am PM / 2 pm ET

Speakers: Edwind Sarmiento, Microsoft SQL Server MVP and Microsoft Certified Master

Max Kolomyeytsev, Product Manager, StarWind Software Inc.

Highly-Available SQL Server with reliable Disaster Recovery is traditionally hard to achieve but necessary to have. We say it is possible to build a fully redundant SQL Server 2014 cluster with just 2 servers, StarWind Virtual SAN and Cluster Shared Volumes while implementing disaster recovery to several places with minimal RTO and RPO!
Join SQL Server MVP and Microsoft Certified Master Edwin M Sarmiento to learn how to use Windows Server Failover Clustering with Cluster Shared Volumes to provide high availability and disaster recovery solutions for mission-critical SQL Server 2014 databases.
We will look at how to configure a highly available SQL Server on top of Cluster Shared Volumes and also discuss best practices and recommendations.

Register here!

3. Free Webinar! Make Your Storage Work For You: VAAI and ODX offload up to 30% of disk operations to the SAN

21_Apr_Webinar_Banner

Date: 21 April, 11am PT / 2pm ET

Speakers: Aidan Finn, Microsoft MVP, Technical Sales Lead at MicroWarehouse Ltd.
Max Kolomyeytsev, Product Manager, StarWind Software Inc.

StarWind always considered virtual storage systems to be valued for uptime, performance and flexibility – take one out and the system is useless. Hypervisor CPU and Networking overhead is a common issue for operations like VM provisioning, migration, and cloning. There should be a way to enhance performance by offloading the unnecessary CPU load and shrinking the SAN traffic. ODX and VAAI features do exactly this by offloading routine disk tasks to the storage array so they will be executed more effectively. That is if your SAN supports one of these features. StarWind Virtual SAN does both – just saying.

Join Aidan Finn and Max Kolomyeytsev at this webinar dedicated to the use of Offloaded Data Transfer (ODX) and APIs for Array Integration (VAAI). They will explain exactly how you can greatly enhance the storage system performance and increase VM density with Hyper-V and vSphere.

Register here!

4. Win the Full Conference Pass for the Microsoft Ignite 2015!

Win_the_MS_Ignite

Attend or register for any ‪‎StarWind ‪‎webinar or ‪#TapRoom Meeting till 16 April and take a chance to win the Full Conference Pass for Microsoft ‪#‎Ignite 2015!

The winner will be announced at StarWind blog, and on Facebook and Twitter pages on the 20th of April!

5. Meet StarWind at MS #Ignite 2015!

Meeting_470_246

Request a chance to schedule a personal meeting with Anton Kolomyeytsev, StarWind Founder and CTO, or Joseph Preston, VP of Sales at Microsoft Ignite 2015

Enjoy!

Cheers,
/Charbel

Posted in Hyper-V, StarWind Tagged with: , , , , ,

Error Applying Virtual Switch Properties Changes. The Requested Resource Is In Use (0x800700AA). #HyperV

Hello folks,

The other day, I came across the following error when I was trying to create a Hyper-V vSwitch using a WiFi network adapter.

HV-WiFi-vSwitch01

As some of you might know that wireless networking is a bit tricky in virtualization, and sometimes does not work.

There is a great post that goes into the needy-greedy details on why supporting VM communication through wireless network adapters is sometimes problematic.

As you can see in the error message above, we don’t have much information to find the root cause of the issue.

Let’s try to create the same Virtual Switch with PowerShell this time.

HV-WiFi-vSwitch06

The error is different now, [The virtual switch cannot be bound to the WiFi adapter because the network bridge is already in use]. No, we are not talking about San Francisco bridge here Winking smile

When you create a Hyper-V vSwitch and bound it to a WiFi Adapter, Hyper-V behind the scene integrates the bridge as part of creating the virtual switch in such a way that when you create an external virtual switch using a WiFi adapter, Hyper-V will do the following:

  1. Create a single adapter bridge connected to the WiFi adapter.
  2. Create the external virtual switch.
  3. Bind the external virtual switch to use the bridge, instead of the WiFi adapter directly.

The error message shows clearly that we have a network bridge already created and bound to the WiFi adapter.

We can confirm this by running: netsh bridge show adapter

HV-WiFi-vSwitch07

Yes indeed we have a network bridge.

Let’s open Network and Connections applet in control panel

HV-WiFi-vSwitch02

Yes, the Network Bridge is already available and connected, but have you noticed something different here?

The Virtual Switch is unplugged, we can confirm this in Hyper-V Manager under Virtual Switch Manager:

HV-WiFi-vSwitch03

We can see the Ethernet External vSwitch, but where is the WiFi vSwitch? Confused smile

Sometimes when you remove the virtual switch that is bound to the WiFi adapter, the network bridge is not deleted automatically.

The resolution is very simple. We will remove the network bridge manually and create the Virtual Switch again:

HV-WiFi-vSwitch09

Here you go, the WiFi vSwitch is created successfully now Smile

HV-WiFi-vSwitch10

However there is a better way to use Hyper-V with a Wireless Network adapter, here is a great post on how to do it documented by Ben Armstrong (Principal program manager on the Hyper-V team).

Hope this helps!

Enjoy your day…

Cheers,
/Charbel

Posted in Hyper-V, Network Tagged with: , , ,

New Webinar: Upcoming features in #HyperV #vNext #Altaro

Are you ready for Hyper-V in Windows Server vNext?

We’re warming up for the release of Hyper-V vNext… My fellow Microsoft Hyper-V MVP, Aidan Finn and Microsoft Sr. Technical Evangelist Rick Claus will take you through what’s coming up in a new Altaro webinar, hosted by my fellow MVP Andy Syrewicze!

Looking to get a good view of what’s coming up? Do you have Hyper-V vNext questions you’d like to ask Aidan or Rick directly?

Here’s your chance! We’re live on Thursday, 26th March 2015 at 4PM CET  / 11am EDT / 8am PDT (30-45mins + live Q&A!)

HyperV-vNext

Enjoy the webinar!

Cheers,
/Charbel

Posted in Altaro, Hyper-V Tagged with: , , ,

Microsoft MVP Profile

About Me

Microsoft Ignite

MVP V-Conference

Sponsor – ALTARO

Sponsor – StarWind

PowerShell Studio

Technical Reviewer

Visitors!

Thanks for visiting!

© Copyright Warning!

All material is copyrighted by me or by its respective owners. To use any of it, full or in part, you must contact me or owner of the material. You may quote few paragraphs from this blog only if you link to the original blog post.
ALL RIGHTS RESERVED ©.
Follow

Get every new post on this blog delivered to your Inbox.

Join other followers: