In October 2010 HP published a customer advisory, warning for cluster resource failures in large Microsoft Windows 2008 and R2 clusters using multiple host NICs with HP P4000 SAN and its Device Specific Module (DSM) for MPIO.
If any combination of cluster nodes, MPIO NIC ports and storage nodes resulted in more than 31 iSCSI sessions per volume, these issues would surface. Cluster Validation tests would in fact fail in these configurations. Adding a cluster node or storage node without validation would fail or only partly work.
HP published a firmware update with patch 10085-00 increasing the number of iSCSI sessions from 31 to 64. HP promised to solve this problem in its next major release of P4000 SAN/iQ software.
The formula for calculating the number of iSCSI sessions is:
# of Microsoft cluster nodes *
( # of initiator NICs per cluster node * # of storage nodes)
Now that SAN/iQ 9.0 has been released we can see that HP has followed up on this issue:
A HP Support document released in December 2010 states that with the new release, it has solved problems with SCSI Persistent Group Reservation (PGR) by increasing the limit to 256 iSCSI sessions per volume. This number is high enough to cope with 16 cluster nodes and 8 storage nodes with two iSCSI network adapters. This adds up to 16 x 2 x 8 = 256: so still be careful with bigger configurations.
Several other improvements have been made to MPIO and iSCSI Session Management. The new DSM supports path load balancing across multiple host network adapters and automatically reduces its sessions per volume to storage systems within the same site to improve load balancing in multi-site SAN configurations.
Native Microsoft MPIO functionality is now supported to allow basic path failover and round robin path load balancing without the need to install additional drivers or components.
Furthermore virtual IP load balancing now dynamically distributes iSCSI sessions across the cluster to maintain cluster wide load balancing when storage nodes are rebooted or added in the cluster. As a bonus, iSCSI sessions are also automatically disconnected when volumes are unassigned from servers to enhance security.