Best Practices to Implement Windows Server 2008 R2 and Hyper-V R2 on HP ProLiant Servers (Part 2)

On the first day of TechEd Europe 2009, Doug Dewerd presented a session on best practices to implement Windows Server 2008 R2 on HP ProLiant Servers. This is in fact the combination of hardware and software I have preferred ever since Windows and ProLiant were joined, exactly 20 years ago last week. The ProLiant server was produced by Compaq and replaced the first PC based server, the Compaq SystemPro which was released in 1989. A nice best practice not many customers are aware of is that ever since the first ProLiant model, the Smart Array Controllers have shared a common driver family. Standardization even on the driver level has worked well for many of our clients. You probably have seen several vendors who introduced new technology, but have switched the configuration software and driver installation methodology quite a bit.

On the agenda were four topics:

• HP Servers and Windows Server 2008 R2
• HP and Hyper-V R2
• HP Management Tools
• HP Storage Support

HP Servers and Windows Server 2008 R2

HP ProLiant Servers combined with R2 mean a considerable reduction in power consumption and improved consolidation ratio’s. The HP ProLiant G6 servers use half the power of previous generations of servers, support more processors and are certified with R2 advanced power management. Especially combined with HP Virtual Connect Flex-10 technology virtualization ratio’s can be improved significantly at a lower price level than traditional switches in a blade enclosure. Furthermore a deep integration with Microsoft System Center improves availability and simplifies management.

There are three interesting things that help efficiency in the datacenter:

1. HP LeftHand SAN products which provide high availability without the requirement to buy additional synchronous replication software. This iSCSI based storage is fully transparent to Hyper-V R2 and Cluster Shared Volumes and Live Migration. LeftHand superbly mirrors data blocks (Network RAID) across storage nodes and by simply spreading the nodes of the storage cluster across sites, a fully automated and disaster tolerant solution can be built. Of course the network interconnects have to be superb as well for synchronous replication to be efficient, although asynchronous replication is also supported out of the box.
2. EVA CLX (Cluster Extensions) now supports Live Migration and disaster recovery. This software, not being the cheapest around, is able to eliminate service disruption and automatically fails over or back with Microsoft failover cluster services when disaster strikes.
3. An integration is made available between the HP Configuration & Sizing Tool for Hyper-V and MAP (Microsoft Assessment & Planning) tool. It can build detailed HP configurations for R2, Hyper-V and applications.

If you want to find out which HP ProLiant Servers are supported for Windows Server 2008 R2 and Hyper-V R2, just go to:
http://h71028.www7.hp.com/enterprise/us/en/servers/ws-servers-2008-r2.html

HP fully supports Windows Server 2008 R2 and Hyper-V R2 for both it’s fifth and sixth generation of tower, rack and blade servers with one or two exceptions. Just check the support matrix.

Power Control with Windows Server 2008 R2

Windows Server 2008 R2 has done great things for intelligent support for power control. R2 supports the new processor performance state interface which enables OS and platform coordination of processor power management:

• Platform is in direct control of t- and p-states
• OS specifies processor performance requirements
• Platform is responsible for delivering requested performance
• Interface is described via ACPI which is jointly developed by HP and Microsoft

The collaborative power controls are available in ProLiant G6 servers and blade and combine the OS independent HP Power Regulator and Windows Server 2008 R2’s Power Metering & Budgeting infrastructure. PMB manages power based on its knowledge of incoming work queues and task scheduling, whilst HP Power Regulator manages power based on the state of the hardware and the reaction speed necessary to protect hardware.

The process is like this:

• Windows asks for what it thinks is best
• The hardware grants that request unless it can’t because of hardware constraints
• Then the hardware tells the OS what it actually got

This joint effort at improving Green IT can reduce energy cost by as much as 70%.

Scalability and Performance

Windows Server 2008 R2 improves on physical processor and memory resources, running exclusively on 64-bit processors, it supports up to a maximum of 256 processors. R2 also improves on virtual processor resources. Hyper-V R2 now supports up to 64 logical processors (cores or hyper-threaded CPU’s). Accepting processor overcommit, Hyper-V R2 allows for 512 virtual CPU’s per physical host. The current limit of vCPU’s per virtual machine is 4.

An extremely important improvement is the support for SLAT (Second Level Address Translation) which reduces hypervisor processor time and saves about 1MB of memory per virtual machine by using processor functionality to carry out virtual machine memory management functions. In short, memory management is delegated to the processor with better scalability and performance as the result.

Doug showed an example of near linear scalability in throughput with no virtual CPU overcommit (n
o more vCPU’s are used than the amount of physical CPU’s available).

These are the graphs you want to see that could make you wonder why you wouldn’t virtualize even your heavier database servers. In fact we only need to get more experience and faith in hypervisors like Hyper-V R2.

A helpful site with technical guidance, reference configurations, performance characterization and test results, white papers, tools, system configurators, capacity planners and sizers can be found on HP ActiveAnswers.

Hyper-V Support

Most ProLiant servers support Hyper-V R2. At a minimum HP ProLiant Support Pack (PSP) 8.30 must be used. Be careful to install PSP before enabling the Hyper-V role and don’t install PSP on the guests. Furthermore, check the HP whitepapers on Hyper-V R2 integration for the minimum ROM version, supported options and NIC Teaming information.

Hyper-V R2 Installation Tips

• Check HP R2 white paper to ensure servers, options and storage are supported
• Update server BIOS if needed
• Enable support for No-Execute and hardware-assisted virtualization in RBSU (Rom Based Server Utility)
• Install OS, then ProLiant Support Pack without Network Configuration Utility (NCU), then enable Hyper-V
  • Â
    • If installing HP Systems Insight Manager or HP Insight Control for System Center, make sure that SNMP is configured and started
• NIC Teaming is supported provided that
  • Â
    • The latest version of Hyper-V is installed and enabled
    • HP ProLiant Network Teaming Software is installed and enabled after having enabled Hyper-V

Hyper-V Server R2 or Server Core with Hyper-V R2 Installation

• Copy PSP executables to subdirectory and extract
• If installing HP SIM or HP Insight Control for System Center, make sure SNMP service has started
  • Â
    • Run oclist.exe to list the services that have started
    • If SNMP has not started, execute the command start /w ocsetup SNMP-SC to start the SNMP service
• Run setup.exe from folder
• Select Bundle Filter and options
• Select items to be installed
• View Installation Results upon completion

NIC Teaming Notes

• Use HP Networking Configuration Utility (NCU) version 9.35 or greater
• VLAN procedure
  • Â
    • Install and enable Hyper-V, and then install the NCU
    • Configure the NIC team(s)
    • Configure VLANs for the team(s)
• IMPORTANT – There is currently no way to easily uninstall HP NCU from Server Core installs
  • Â
    • Can cause a problem when trying to perform Hyper-V updates
    • Perform manual uninstall of HP NCU following http://h20000.www2.hp.com/bc/docs/support/SupportManual/c01663264/c01663264.pdf and http://support.microsoft.com/kb/950792/en-us
    • HP will release a fix in a future version of NCU

Known Hyper-V R2 Issues

• Stop 0x0000007E blue screen displays after initial reboot when Hyper-V is installed on Windows Server 2008 R2
  • Â
    • Microsoft has published a KB Article and a Hot Fix to prevent the issue from occurring at http://support.microsoft.com/kb/974598
• After enabling the Hyper-V server role on an HP ProLiant DL785 G5 Server, a black screen displays after the system reboots
  • Â
    • Upgrade the ProLiant DL785 G5 server to version A15 of the System ROM dated 08/14/2009 (or later).
• The virtual machine network is not working when the external virtual network is connected to 1-Gbps NC375i NIC on an HP ProLiant ML370 G6 Server
  • Â
    • This issue has been reported to HP and is being worked
    • The network driver will be released to the support and drivers page after the issue is resolved

Networking Challenges

• The total network throughput requirement on Hyper-V host is significantly higher than on traditional servers
• Consolidated servers may not have same requirements
  • Â
    • Access to different VLAN’s
    • Access to additional bandwidth
• Varying requirements lead to additional complexities
  • Â
    • Increased number of uplinks
    • Increased number of ports
    • Increased number of cables

 

If you add up all the NICs that are required for all the networks with full redundancy in an ideal configuration, you end up with staggering numbers.

In fact this could be downsized a little in my opinion. Let’s say you use fibre channel SAN and a Hyper-V R2 cluster with Live Migration, I would normally advise:

In previous Windows failover clusters there was a requirement for a Cluster Heartbeat (HB) or Private Network. In Windows Server 2008 Failover Clusters, an alternative path is recommended so if one NIC goes down, another NIC is used to check the cluster heartbeat. In our example the first two networks can serve this purpose. Since Windows Server 2008 the heartbeat/cluster NIC is supported in a NIC Teaming configuration.

For production VM’s at least two teamed NIC’s can be used with at least 2 x 1GB speed. Maybe even 4 x 1GB speed to allow for more network throughput. If 8 NICs are used we end up with eight Gigabit switches in a typical HP c-Class 7000 blade enclosure. Counting these switches including, cabling and core Ethernet switch ports plus tedious management, this can easily become a very expensive solution. For this reason HP has developed HP Virtual Connect Flex-10 Interconnects. With only two of these modules, each BL ProLiant BL G6 and higher server can be configured to split the two 10Gb Ethernet ports into 8 so called FlexNICs which are not virtual but fully functional hardware based Ethernet NICS. The speed of these FlexNICs can be set to any speed between 100MB and 10Gb Ethernet.

With Flex-10 the table looks like this:

If more is required in the future with even higher densities of cores, memory and consequently VM’s, we can add up to six HP VC Flex-10 Interconnect modules with the total capacity of 24 NICs per server blade. Who said blade servers are not scalable? Unfortunately I cannot tell you what HP showed us in the NDA room, but servers are surely bound to become bigger and much more scalable. Sorry, you have to wait until some time next year. But rest assured that the current HP ProLiant G6 family is very scalable and even half-height blades can handle up to 24 cores, 192GB of memory and 4 x 10Gb of network capability (if you include an extra dual-port 10Gb Ethernet mezzanine module. Of course scalability will improve further in the next year. And you can grow with it, while keeping the server blade infrastructure in place.

HP Insight Portfolio

HP offers several management products for its ProLiant servers: HP Insight Foundation (included), HP Insight Control and HP Insight Dynamics.

The best news for HP and Hyper-V users is the tight integration of HP Insight software with Microsoft System Center:

• Provides seamless integration of the unique ProLiant and BladeSystem manageability features for customers who have standardized on System Center management platforms
• Delivered as part of HP Insight Control 6.0
  • Â
    • In addition to the full Insight Control capabilities, it provides a set of extensions to Microsoft System Center
    • Requires one Insight Control license per managed (Hyper-V R2) server

Integration Components

• HP Insight Control for System Center software CD components
  • Â
    • HP ProLiant Server Management Packs for Operations Manager 2007
    • HP BladeSystem Management Pack for Operations Manager 2007
    • HP ProLiant PRO Management Pack for Virtual Machine Manager 2008 (R2?)
    • HP ProLiant Server OS Deployment for Configuration Manager 2007
    • HP Hardware Inventory Tool for Configuration Manager 2007
• HP Catalog for System Center Configuration Manager is downloaded from the HP web
  • Â
    • No HP Catalog software is included on the CD
    • HP Catalog support included with standard Technical Support and Software Update service

Storage Support for Hyper-V R2

Check the support matrix for HP storage and Hyper-V R2. The tables show extensive support for the current Microsoft hypervisor. In addition to this also check the Single Point of Connectivity Knowledge (SPOCK) website. You have to create or use an HP Passport for access.

One of the first storage products that fully support Hyper-V R2, Cluster Shared Volumes and Hyper-V R2 is HP LeftHand P4000 SAN Solution. Notably the Network RAID and replication capability (included in the price) is responsible for the transparency for the Hyper-V R2 cluster. A multi-site cluster of course requires a good design and proper network setup, but HP LeftHand takes a lot of complexity out of multi-site clustering.

With storage that is optimized for Hyper-V virtualization and Live Migration, virtual machines can be moved live without downtime for users. Users will not notice that both the server and the storage is moved to another datacenter. When a datacenter breaks down or blows up, virtual machines will take over in a very short amount of time (minutes), comparable to a server crash. The virtual machine is restarted quickly and users can continue their work as if nothing had happened.

Other possibilities:

• Zero Downtime Array Load Balancing
  • Â
    • IOPS
    • cache utilization
    • power consumption
• Zero Downtime Maintenance
  • Â
    • Firmware
    • HBA
    • Server updates without user interruption
• Follow the sun/moon data center access model
  • Â
    • Move the app/VM closest to the users or closest to the cheapest power source
• Failover, failback, Quick and Live Migration using the same management software
  • Â
    • No need to learn x different tools and their limitations

HP also showed us an update to HP Cluster Extensions (CLX) for Hyper-V R2 and Live Migration. This solution is intended for HP EVA and HP XP Storage Arrays with multiple sites and synchronous replication (Continuous Access). Because of the architecture of HP Continuous Access it currently cannot use Cluster Shared Volumes (CSV), which is not a problem because Live Migration does formally not depend on CSV. HP CLX therefore uses traditional shared LUNS per VM. In the demo we saw only a few pings lost during a Live Migration of both virtual machine and storage. No packet had to be retransmitted and users would not be aware of the datacenter move.

From this presentation by Doug Dewerd and Matthias Popp, I got the impression that HP takes Microsoft and Hyper-V very seriously. They have worked together for many years in the Microsoft/HP Frontline Partnership which has now culminated into fascinating products which work really well together.

With the recently formed vBlock called VCE (VMware, Cisco and EMC), I contend that HP is sort of forming an opposing block. I propose calling them MCH (Microsoft, Citrix and HP).

In fact this combination of best of breed server, storage and virtualization solutions happens to be my primary job: designing and architecting fast, flexible, highly available and highly scalable infrastructures. I got a feeling …

For Hyper-V R2 and HP Full Featured DSM / MPIO also see:
https://hyper-v.nu/blogs/hans/?p=292