How to Deploy Websense In Stand-alone Mode on a Hyper-V Virtual Machine? #HyperV #Websense

Hello folks,

Since the initial release of Hyper-V back in 2008, my hope was to move all the physical workloads to Hyper-V, however one of the role that was impossible to move is Websense Web Security!

Websense Web Filter and Security blocks web threats to reduce malware infections, decrease help desk incidents and free up valuable IT resources. More information on Websense.

Since then, I had several conversation with Websense and Microsoft folks and all the feedbacks came negative Sad smile

So what is the reason that Websense cannot run on Hyper-V virtual machine?

Websense feedback was that Hyper-V server does not allow setting a virtual NIC (vNIC) to promiscuous mode, Websense does not certify and support the platform. Websense Network Agent requires a NIC set to promiscuous/stealth mode. As Microsoft does not intend to support this feature, Network Agent cannot successfully monitor traffic from other systems on its monitoring NIC.

If Websense is working with an integration mode other than Network Agent (Standalone mode), where port spanning is not necessary, then Hyper-V is a viable platform. The integration mode like Threat Management Gateway (TMG). While TMG has been officially deprecated by Microsoft, and it is still supported until April of 2020.

The high level architecture of standalone mode deployment in the physical world is showing in below figure:

HV-Websense02

Microsoft feedback was, Hyper-V virtual switch does not offer promiscuous mode on its virtual interfaces.

In short, promiscuous mode allows a network device to intercept and read each network packet that arrives in its entirety. The most typical use cases include network intrusion detection systems (NIDS), monitoring tools such as (Wireshark, Microsoft Message Analyzer, etc.), web security tools such as Websense or recording of calls in VOIP-based centers such as MiaRec. This mode of operation is given to a network server that captures and saves all packets for analysis.

The Websense deployment and configuration is already well explained elsewhere, so I’ll keep the basics to Hyper-V here.

VMware does support promiscuous mode, but I believe in Hyper-V Winking smile, so what is the solution?

Microsoft in Windows Server 2012/2012 R2 Hyper-V introduced the concept called (port mirroring) which can be enabled on any virtual machine vNIC adapter. There are quite some official documentation available if you want to setup port monitoring between two or more Virtual Machines.

How does port mirroring work?

Port Mirroring allows you to monitor virtual network traffic from one or many virtual machines (sources) to another virtual machine (destination). Port Mirroring works at the Virtual Switch level and to be precise, it’s the Hyper-V virtual switch extension capabilities that is being used to achieve port mirroring/capturing. The extensible vSwitch by itself uses port ACLs to set a rule that forwards traffic from one vNIC in a VM to another vNIC in another VM.

For example if we need to monitor all traffic sent and received by both VM1 and VM2, you can run the following PowerShell cmdlets where MonitorVM has a network monitoring tool installed i.e. Wireshark in order to capture the traffic.

This is a great feature for internal traffic between VMs on the same physical host, but this does not solve my pain point with Websense, because we need to be able to monitor the traffic from a port on the physical switch to a virtual port (vNIC) inside a VM.

What about external traffic?

Hyper-V does not support to set a “promiscuous mode” flag on a virtual port, as you need to specify if a given port is supposed to be the source or the destination of the network packets.

The interesting part is that the official documentation does not state that you can also capture traffic from an external network or from the host parent partition. Waw this is what is needed in my scenario.

The Hyper-V Extensible Switch and the PowerShell module have the bells and whistles to tackle this problem.

What are the requirements to capture external traffic?

1. Two vNICs To Websense VM (Block NIC and Monitoring NIC).
2. Set the Mirroring mode of Websense VM monitoring vNIC to “Destination“.
3. Enable Microsoft NDIS Capture on the Hyper-V Virtual Switch Extensions where Websense VM is attached to.
4. Set the Mirror mode on the External port of the Hyper-V Virtual Switch where Websense VM is attached to reflect as the “Source“.
5. Configure port mirroring on the physical switch to mirror any traffic on your firewall/router port ==> to the port that Hyper-V server is connected to.

Step 1: Add Two Virtual NICs To Websense VM

HV-Websense03a

Step 2: Set The Mirroring Mode Of  The Monitoring Virtual NIC To “Destination”

HV-Websense03b

The same can be done in Hyper-V Manager

HV-Websense04

Step 3: Enable Microsoft NDIS Capture Extension On The Virtual Switch

  1. Open the Virtual Switch Manager on the Hyper-V Host.
  2. Expand the virtual switch name “Mirroring_VM_vSwitch” and click on “Extensions“.
  3. Enable Microsoft NDIS Capture under Extensions.

HV-Websense05

Step 4: Set The Mirror Mode On The External Port Of The Virtual Switch To “Source”

The Hyper-V PowerShell module includes the following cmdlets (Add-VMSwitchExtensionPortFeature, Get-VMSystemSwitchExtensionPortFeature, Remove-VMSystemSwitchExtensionPortFeature, and Set-VMSystemSwitchExtensionPortFeature) that can be used to manage port monitoring at the host level.

We need to configure the Hyper-V vSwitch name “Mirroring_VM_vSwitch” that any traffic hits the external port “SOURCE”, has to be forwarded to the vNIC “Monitor” that we configured “DESTINATION” on Websense VM.

The following PowerShell cmdlets will help you to set the External vSwitch port to “SOURCE” Mirror mode:

Let’s validate the Monitoring mode is set to “SOURCE” by running the following cmdlet:

HV-Websense09

MonitorMode=2 is “SOURCE“, MonitorMode=1 is “DESTINATION“, and MonitorMode=0 is “NONE

Once your run the above cmdlets on the Hyper-V host, all traffic passing on the external NIC of Mirroring_VM_vSwitch will be “mirrored” to Websense VM which port monitoring mode has been set to destination.

Step 5: Configure Port Mirroring On The Physical Switch

HV-Websense06

In my demo, I am mirroring the traffic to two destination NIC interfaces where the Hyper-V host is connected to, because I am using NIC Teaming on the host and the “Mirroring_VM_vSwitch” is created on top of the team.

As soon as you start mirroring the traffic to the Hyper-V host, you can open Websense VM and observe the received traffic on the mirroring vNIC.

HV-Websense08

Once the above steps are followed, you should be able to start filtering the happy users Smile:

HV-Websense07

What are the best practices?

1. Have a separate dedicated physical NIC or team NICs on the host.
2. Have a separate external vSwitch, because Websense VM will be always available and you don’t want to flood your existing production vSwitch.

Closing thoughts

Keep in mind that all this works within the boundaries of the same physical host. Which means that if you want to move Websense VM across nodes in a cluster or to a different host, you need to configure step 3, 4 and 5 above on each node separately with the same virtual switch name. This means that when Websense VM is live migrated to a second node, it will continue monitoring the traffic. That works!

Happy filtering day!

Cheers,
-Charbel

About Charbel Nemnom 323 Articles

Charbel Nemnom is a Microsoft Cloud Consultant and Technical Evangelist, totally fan of the latest’s IT platform solutions, accomplished hands-on technical professional with over 15 years of broad IT Infrastructure experience serving on and guiding technical teams to optimize performance of mission-critical enterprise systems. Excellent communicator adept at identifying business needs and bridging the gap between functional groups and technology to foster targeted and innovative IT project development. Well respected by peers through demonstrating passion for technology and performance improvement. Extensive practical knowledge of complex systems builds, network design and virtualization.

16 Comments

  1. I’ve followed the guide and am running into issues. I have set up everything but the block nic. I thought this would just be the main NIC the server uses. Do I need to have 3 NICs? 1 Monitor, 1 block, and 1 actual uplink?

    • Hello Preston,

      Thanks for the feedback.

      I strongly recommend to have a dedicated Physical NIC on the Hyper-V Host used for Websense.
      As for Websense VM, you need two vmNICs, one is used for the MGMT with IP to send block pages to the users and the second one is used for traffic monitoring.
      The Physical NIC on the host is bound to a new Virtual Switch and then follow the steps in this article to set the VM Switch Extension Port Feature.

      Hope this helps!

      Cheers,
      ~Charbel

      • Thanks. I do have a dedicated NIC for the uplink to the network. Then I added one NIC for monitoring which is connected to the physical switch on a span port. Judging by your screenshots it looks like you connected the block NIC just to the regular uplink, so I understood this to mean that you were using the regular NIC for blocking as well.

        I opened up a Microsoft support request and they suggested that the setup would not work due to the block NIC and monitoring NIC being connected to different vSwitches. Where am I misunderstanding? 🙁

        • Hello Preston,

          As you can see in my screenshot for Websense VM settings. I have two different vmNICs, the first vmNIC is attached to Team vSwitch that I am using for all VMs on that host, this vmNIC has the MGMT IP address for Websense, and the second vmNIC is attached to the dedicated Websense vSwitch set in Mirroring mode as source, this vmNIC inside the VM is set without IP address, in other words the IPv4 and IPv6 (TCP/IP) Protocols are deselected.

          I have the same deployment in more than 10 sites and it’s working perfect.

          Hope this help.

          Cheers,
          ~Charbel

          • So the monitor NIC is set as source? I am set up exactly like that (I’ve actually flipped Source and Destination around to test that way as well..), but see no traffic on the second NIC with IPv4 and IPv6 disabled.

            I can plug a laptop into the span port and see all the traffic with no issues..

          • Hello Preston,

            The monitor vmNIC in Websense VM settings is set to Destination, and the Virtual Switch is set to Source, please refer to the article again.

            One important point to mention, you will not see traffic inside the Guest OS (Websense) until you finish Websense configuration and the filtering service is up and running correctly (Network Agent, Block NIC, etc…).

            Hope this helps.

            Cheers,
            ~Charbel

  2. When you say that I won’t see traffic until Websense is configured, does that mean even if I go into the control panel and look at the NIC I won’t see any traffic?

    • Yes, even if you go into control panel (Network and Sharing Center) and you open the vNIC, you won’t see any traffic (received traffic actually and not sent) until you configure Websense / Physical Switch with port mirroring.
      However in the physical world this is different.
      If you check the Physical NIC on the host that is assigned to Websense Virtual Switch, you will see incoming traffic.

  3. Hi Charbel,

    I am setting up a VM within Hyper-V 2012 R2 which following your guide sees traffic but it appears to be only one direction (inbound) any ideas what would cause this on Hyper-V?

    • Hello Colin,

      This is normal for mirroring NIC, you will see incoming traffic (Receive) on the mirroring NIC and not send.
      The second vmNIC which is configured with IP address will send the block pages to the users.

      Could you please describe more your setup?

      Cheers,
      ~Charbel

  4. Hi Charbel,

    Thanks for this guide, much appreciated, I’ve followed through this but at the 3rd step of configuring the source port/NIX I do receive this error..

    PS C:\> Add-VMSwitchExtensionPortFeature -ExternalPort -SwitchName SWITCHPORTNAME -VMSwitchExtensionFeature $ExtPortFeature

    Add-VMSwitchExtensionPortFeature : Failed while modifying virtual Ethernet switch connection settings.
    At line:1 char:1
    + Add-VMSwitchExtensionPortFeature -ExternalPort -SwitchName SPANPORT -VMSwitchExt

    I have a VM with two NIC’s as you do, one management/blocking NIC and the other set up as the destination/monitoring NIC. On the host I can see ALL traffic on the dedicated physical NIC adapter which is bound to my virtual monitoring NIC…however the traffic isn’t passing through to the vNIC.

    Your guide has given me hope with the Server 2012 port mirroring feature because other sites I’ve visited indicate that Server 2012 mirroring is purely for monitoring between VM’s and not for monitoring traffic on a physical network using a VM.

    Thanks
    Stuart

    • Hello Stuart,

      Are you running the following cmdlets as Administrator?

      $ExtPortFeature=Get-VMSystemSwitchExtensionPortFeature -FeatureName “Ethernet Switch Port Security Settings”
      $ExtPortFeature.SettingData.MonitorMode = 2
      Add-VMSwitchExtensionPortFeature -ExternalPort -SwitchName Mirroring_VM_vSwitch -VMSwitchExtensionFeature $ExtPortFeature

      You won’t see traffic passed into the vmNIC inside the VM unless you enable and start the Network Agent for Websense.

      I have the same setup deployed in multiple sites and it’s working beautifully.

      Hope this helps.
      -Charbel

  5. Ignore that, got it working! removed all previously created switches and NIC’s and stepped through your guide again…also had to reinstall Network Agent to pick up the new NIC’s but fine now.

    Thank you very much, I’m happy, not sure my users will be 😉

    Stuart

  6. I would like to know if this would work for pfSense?

    It seems like it would.

    I have a server with two physical NICs. One is for WAN and the other is for LAN. Does your method mean I need to add a third physical NIC to the server?

    How is the configuration of your vSwitches done?

    • Hello Zane,

      Please follow the number of NICs needed based on pfSense requirements.
      The configuration for the vSwitches are done as documented in this post.
      Yes, it should work with any product that requires promiscuous mode.

      Cheers,
      Charbel

Leave a Reply