hp-blade

1
Technical white paper
iSCSI Cookbook for HP Virtual Connect
Version 4.20 Firmware Enhancements
May 2014


Table of contents
Purpose 6
Introduction 6
System requirements 7
HP FlexFabric Adapters Support 7
Virtual Connect Modules Support 7
Virtual Connect with iSCSI Summary
support 8
Firmware and Software Support 9
iSCSI Storage Target Support 9
Networking recommendations 11
Network considerations 11
Keep it simple and short 11
Flow Control recommendations 12
Enabling Flow Control on iSCSI Storage
Systems 12
Enabling Flow Control on Network
Switches: 12
Enabling Flow Control on Virtual Connect: 13
Enabling Flow Control on Host Accelerated
iSCSI adapters 13
Jumbo Frames (optional
recommendation) 13
Enabling Jumbo Frames on iSCSI Storage
Systems 13
Enabling Jumbo Frames on Network
Switches 14
Enabling Jumbo Frames on Virtual Connect 14
Enabling Jumbo Frames on Hosts
Accelerated iSCSI adapters 14
iSCSI multipathing solutions 21
Virtual Connect network scenarios 22


2
Scenario 1: iSCSI network physically
separated 23
Defining two iSCSI networks vNet 25
Scenario 2: iSCSI network logically
separated 27
Defining a first Shared Uplink Set (VLAN-
trunk-1) 29
Defining a second Shared Uplink Set
(VLAN-trunk-2) 30
Scenario 3: Direct-attached iSCSI
Storage System 31
Direct-attached Limitations 31
Scenario 3-A: Direct-attached iSCSI device
with out-of-band management (using
separate ports for management and host
traffic) 32
Scenario 3-B: Direct-attached iSCSI device
with in-band management (using the same
ports for management and host traffic): 34
Scenario 3-C: “Low cost” Direct-attached
iSCSI device with or without out-of-band
management 35
Virtual Connect Network configuration 36
Connecting the Direct-Attached iSCSI SAN
to the VC Domain 37
Preparing the network settings of the
storage system 38
Configuring the Storage Management
Server 38
Configuring the network interface bonds
on the Storage System 40
Configuring the iSCSI Host 42
Accelerated iSCSI 43
Creating a Virtual Connect profile with
Accelerated iSCSI 45
Accelerated iSCSI with Microsoft
Windows Server 48
Installation of the vendor’s adapter utility 48
IP Configuration of the iSCSI HBA ports 52
Enabling jumbo frames and Flow Control
on the NIC adapters 54
Installation of the Microsoft iSCSI initiator 55
Connecting volumes with the Microsoft
iSCSI Initiator 57
Installation of Microsoft MPIO 60


3
Claim an iSCSI-attached device for use with
MPIO 60
Installation of the Device Specific Module
(DSM) 63
Using the Microsoft iSCSI Software Initiator
in conjunction with Accelerated iSCSI
support 68
Accelerated iSCSI with VMware vSphere
Server 76
Installing iSCSI drivers of the FlexFabric
adapter 80
Installing the vendor’s FlexFabric Adapter
utility (optional) 81
Installing Broadcom FlexFabric dependent
hardware iSCSI adapters 82
Installing Emulex FlexFabric independent
hardware iSCSI adapters 92
Using the VMware iSCSI Software Initiator
in conjunction with Accelerated iSCSI
support 96
Boot from iSCSI 111
Boot from iSCSI : Creating a Virtual
Connect profile 111
Boot from iSCSI: Installing Microsoft
Windows Server 2012 120
Boot from iSCSI: Installing Microsoft
Windows Server 2008 125
Boot from iSCSI: Installing VMware
vSphere 5.x 130
Boot from iSCSI: Installing Red Hat
Enterprise Linux 134
Boot from iSCSI: Installing SUSE Linux
Enterprise Server 135
Troubleshooting 136
Emulex iSCSI Initiator BIOS Utility 136
Configuration checking 138
Broadcom iSCSI Initiator BIOS Utility 141
Emulex OneCommand Manager (OCM) 144
Broadcom Advanced Control Suite
(BACS) 147
Graphical User Interface 147
Problems found with OneCommand
Manager 150


4
Problems found during iSCSI Boot 151
PXE booting problems 156
iSCSI boot install problems with
Windows Server 157
VCEM issues with Accelerated iSCSI
Boot 158
iSCSI issues with HP StoreVirtual 4000
Storage 158
Appendix 1- iSCSI Boot Parameters 160
Mandatory iSCSI Boot Parameters
entries 160
iSCSI Initiator (iSCSI Boot Configuration) 161
iSCSI Target (iSCSI Boot Configuration) 162
Initiator Network Configuration 163
Optional iSCSI Boot Parameters entries 164
Secondary iSCSI Target Address 164
Security enhancement using an
authentication 164
Appendix 2 - Dynamic configuration of
the iSCSI Boot Parameters 167
Windows 2008 DHCP server
configuration 168
Linux DHCP server configuration 176
Format of DHCP option 43 for the
Emulex FlexFabric Adapters 177
Description of Optional Parameters 177
Examples 178
Example with target and initiator name: 178
Default Initiator name and Data Digest
settings: 178
Default TCP Port and Mutual CHAP: 178
Appendix 3- How to monitor an iSCSI
Network? 179
Monitoring Disk Throughput on the
iSCSI Storage System 179
Monitoring Network and Disk
Throughput on the iSCSI Host 180
VMware vSphere 180
Microsoft Windows Resource Monitor 182
Analyzing Network information from
the Virtual Connect interface 183


5
Analyzing Virtual Connect Network
performance 187
Wireshark 189
Software iSCSI analysis 189
iSCSI analysis for Accelerated iSCSI
adapters 191
Acronyms and abbreviations 194
Support and Other Resources 195
Contacting HP 195
Before you contact HP 195
HP contact information 195
Subscription service 195
Related documentation 196




6
Purpose

The purpose of this Virtual Connect iSCSI Cookbook is to provide users of Virtual Connect with a better understanding of
concepts and steps required when using iSCSI with Virtual Connect components. This document will help users to answer
some of the typical questions on iSCSI: What are the network considerations to properly build an iSCSI network? What
are the components supported by HP? How can I troubleshoot my iSCSI environment?
In addition, this document describes some typical iSCSI scenarios to provide the reader with some valid examples of how
HP Virtual Connect with iSCSI could be deployed within their environments.
Tips and some troubleshooting information for iSCSI boot and install are also provided.
Detailed information regarding Emulex requirements is subject to change, and readers should always refer to the
documentation from the providers.

Introduction

The iSCSI standard implements the SCSI protocol over a TCP/IP network. While iSCSI can be implemented over any TCP/IP
network, the most common implementation is over 1 and 10 Gigabit Ethernet (GbE). The iSCSI protocol transports block-
level storage requests over TCP connections. Using the iSCSI protocol, systems can connect to remote storage and use it
as a physical disk although the remote storage provider or target may actually be providing virtual physical disks. iSCSI
serves the same purpose as Fibre Channel in building SANs, but iSCSI avoids the cost, complexity, and compatibility
issues associated with Fibre Channel SANs.
Because iSCSI is a TCP/IP implementation, it is ideal for new field deployments where no FC SAN infrastructure exists. An
iSCSI SAN is typically comprised of software or hardware initiators on the host connected to an isolated Ethernet
network and some number of storage resources (targets). While the target is usually a hard drive enclosure or another
computer, it can also be any other storage device that supports the iSCSI protocol, such as a tape drive. The iSCSI stack
at both ends of the path is used to encapsulate SCSI block commands into Ethernet Packets for transmission over IP
networks.
An interresting technology of iSCSI is the iSCSI boot, it allows servers to boot from an operating system image located on
a remote iSCSI target. iSCSI Boot enables organizations to purchase less expensive diskless server, to provide rapid
disaster recovery and a more efficient usage of the storage, etc.
Another nice feature is the accelerated iSCSI that can also be enabled with the HP FlexFabric adapters, it offloads the
iSCSI function to the Converged Network Adapter rather than taxing the CPU of the server.




7
System requirements

When using HP Virtual Connect technology, iSCSI Boot and Accelerated iSCSI are only supported with the following
components:


HP FlexFabric Adapters Support
 Integrated NC553i Dual Port FlexFabric 10Gb Adapter (Intel based BladeSystem G7 servers)
 Integrated NC551i Dual Port FlexFabric 10Gb Adapter (AMD based BladeSystem G7 servers)
 HP NC551m Dual Port FlexFabric 10Gb Converged Network Adapter
 HP NC553m Dual Port FlexFabric 10Gb Converged Network Adapter
 HP FlexFabric 10Gb 2-port 554FLB / 554M Adapter
 HP FlexFabric 10Gb 2-port 534FLB / 534M Adapter
 HP FlexFabric 20Gb 2-port 630FLB / 630M Adapter


Note: iSCSI Boot is available as well on Virtual Connect with the QLogic QMH4062 1GbE iSCSI Adapter but with some
restrictions. The QMH4062 iSCSI settings cannot be managed by a Virtual Connect profile but they can manually be set
through the Qlogic Bios (CTRL+Q during Power-on Self-Test). The constraints to remember is that during a Virtual
Connect profile move, the iSCSI boot settings will not be saved and reconfigured on the target server.


Virtual Connect Modules Support
 HP Virtual Connect FlexFabric-20/40 F8 Module
 HP Virtual Connect FlexFabric 10Gb/24-port Module
 HP Virtual Connect Flex-10/10D Module
 HP Virtual Connect Flex-10 10Gb Ethernet Module



Note: 10Gb KR-based Ethernet switches (like Procurve 6120XG, Cisco 3120G) can be used as well for Accelerated iSCSI
boot but this is not covered in this document
NC553m
NC551m
m
554FLB 534M 630M


8
Virtual Connect with iSCSI Summary support

When using HP Virtual Connect technology, iSCSI Boot and Accelerated iSCSI is only supported with the following
combination of devices:

BladeSystem Gen8


+

Virtual Connect FlexFabric 10Gb/24-port
or FlexFabric-20/40D-F8

BladeSystem Gen8 with
FlexFabric 10Gb 2-port 554FLB/554M or
FlexFabric 10Gb 2-port 534FLB/534M or
FlexFabric 20Gb 2-port 630FLB/630M




+

Virtual Connect Flex-10 or Flex-10/10D


BladeSystem Gen8 with
FlexFabric 10Gb 2-port 554FLB/554M or
FlexFabric 10Gb 2-port 534FLB/534M or
FlexFabric 20Gb 2-port 630FLB/630M



BladeSystem G7


+

Virtual Connect FlexFabric 10Gb/24-port
or FlexFabric-20/40D-F8

BladeSystem G7
with NC551i/NC553i Integrated CNA or
NC551m/NC553m FlexFabric Adapter




+

Virtual Connect Flex-10 or Flex-10/10D


BladeSystem G7
with NC551i/NC553i Integrated CNA or
NC551m/NC553m FlexFabric Adapter





9
BladeSystem G6


+

+

Virtual Connect FlexFabric 10Gb/24-port
Support for FlexFabric-20/40D-F8

NC551m / NC553m
10Gb 2-port FlexFabric
Adapter

BladeSystem G6
Latest System BIOS



+

+

Virtual Connect Flex-10 or Flex-10/10D


NC551m / NC553m
10Gb 2-port FlexFabric
Adapter


BladeSystem G6
Latest System BIOS


Note: HP BladeSystem c-Class Integrity Server Blades do not support Accelerated iSCSI and iSCSI boot with Virtual
Connect.


Firmware and Software Support
HP recommends to use the latest Service Pack for Proliant, for more information, see
http://h18004.www1.hp.com/products/servers/service_packs/en/index.html
Requirements for Accelerated iSCSI and iSCSI boot:
 Virtual Connect 3.10 is the minimum VC Firmware required for Accelerated iSCSI and iSCSI boot
 With Emulex adapters:
 be2iSCSI driver (Offload iSCSI driver)
 be2iSCSI Driver Update Disk for Emulex iSCSI boot installs
 OneCommand Manager (recommended)
 With Broadcom adapters: Offloaded iSCSI operations require both the networking driver and the bnx2i driver.
 bnx2i driver (Offload iSCSI driver) + bnx2x (networking driver)
 Broadcom Advanced Control Suite (recommended)
 iSCSI target
 DHCP server (optional)

Note: If the VC firmware is downgraded to a version older than 3.10, the iSCSI boot parameter configuration is not
supported and all iSCSI boot parameters are cleared.


iSCSI Storage Target Support
Hardware and software based iSCSI initiators support for a specific HP Storage Arrays should be checked by using HP's
SPOCK portal, see http://www.hp.com/storage/spock.
SPOCK is the HP authoritative source of interoperability information for HP storage products. A particular configuration
is supported if and only if, it is listed on SPOCK.


10
For example with StoreVirtual/LeftHand targets, go to
http://h20272.www2.hp.com/Pages/spock2Html.aspx?htmlFile=hw_lefthand.html, select the StoreVirtual Storage
Compatibility Matrix and make sure either your iSCSI Software initiator or iSCSI Hardware initiator (i.e. the server’s
FlexFabric adapter) are listed.
For information about Non-HP Storage support, firmware versions and specific supported configuration information,
consult your storage equipment vendor.





11
Networking recommendations

Network considerations
When constructing an iSCSI SAN with HP Virtual Connect, some network considerations must be taken into account.
Don’t think of iSCSI network as just another LAN flavor, IP storage needs the same sort of design thinking applied to FC
infrastructure particularly when critical infrastructure servers can boot from a remote iSCSI data source.
Network performance is one of the major factors contributing to the performance of the entire iSCSI environment. If the
network environment is properly configured, the iSCSI components provide adequate throughput and low enough
latency for iSCSI initiators and targets. But if the network is congested and links, switches or routers are saturated, iSCSI
performance suffers and might not be adequate for some environments.
Here are some important tips and tricks to think about:


Keep it simple and short
With iSCSI, it is possible to route packets between different networks and sub networks but keep in mind every route and
hop a packet must use, network latency will be added and affect tremendously the performance between iSCSI initiator
and iSCSI target.
A network switch will also add latency to the delivery time from the iSCSI packet, so we recommend keeping the distance
short and ovoid any router or network switches in between the connection. It simply costs performance, reduces the
IOP’s per second and the chances of storage traffic competing with other data traffic on congested inter switch links.
To avoid bottlenecks, inter switch links should be sized properly and use stacking cables, 10-Gigabit Ethernet uplinks, or
link aggregation or port trunking.
Networking considerations include:
 Minimizing switch hops
 Maximizing the bandwidth on the inter-switch links if present
 Use of 10G Ethernet uplinks




12
Flow Control recommendations

Storage vendors usually have iSCSI SAN design recommendations; enabling Flow Control is one of the most important
ones.
Ethernet Flow control is a mechanism to manage the traffic flow between two directly connected devices and makes use
of pause frames in order to notify the link-partner to stop sending traffic when congestion occurs. It helps resolve in an
efficient manner any imbalance in network traffic between sending and receiving devices.
Enabling Flow Control is highly recommended by iSCSI storage vendors. It must be enabled globally across all switches,
server adapter ports and NIC ports on the storage node.

Enabling Flow Control on iSCSI Storage Systems

Flow Control can usually be enabled on all iSCSI Storage Systems. For more specific information about how to enable
Flow Control, see the Storage System’s manufacturer documentation.

On a HP StoreVirtual 4000 Storage, Flow Control can be set from the TCP/IP settings page within the CMC console:


Enabling Flow Control on Network Switches:

Flow Control should be enabled on every switch interfaces connected to the Storage device.
See the switch’s manufacturer documentation for more information about Flow Control.
Note: On ProCurve switches, if the port mode is set to "auto" and flow control is enabled on the HP StoreVirtual 4000
Storage port, the switch port will auto-negotiate flow control with the Storage device NIC.





13
Enabling Flow Control on Virtual Connect:

Flow Control is enabled by default on all downlink ports. To enable Flow control on all VC ports, including uplink ports,
enter:
-> set advanced-networking FlowControl=on

Note: Be careful, this command can result in data traffic disruption!


Enabling Flow Control on Host Accelerated iSCSI adapters

There is no configuration required as by default, Flow Control is enabled on every network interfaces when Accelerated
iSCSI is enabled.
Note: For Software iSCSI, it might be necessary to enable Flow Control at the NIC/iSCSI initiator level.



Jumbo Frames (optional recommendation)

Jumbo frames (MTU>=9000 bytes) are also frequently recommended by iSCSI storage vendors as it can significantly
increase the iSCSI performance.
Jumbo Frames have many benefits particularly for iSCSI traffic as it reduces the fragmentation overhead which
translates straight away to lower CPU utilization. It gives as well more aggressive TCP dynamics, leading to greater
throughput and better response to certain types of loss.
Jumbo frames must be correctly configured end-to-end on the network, from the storage to the Ethernet switches and
up to the server ports.
Note: Usage of the Jumbo frames in some environment can cause more problems than helps with performance. This is
frequently due to misconfigured MTU sizes but also because some devices support different max MTU sizes. So if you are
unsure about whether your routers and other devices support larger frame sizes, keep the frame size at the default
setting.


Enabling Jumbo Frames on iSCSI Storage Systems

Jumbo frames can usually be used with all iSCSI Storage Systems and are mostly of the time enabled by setting the MTU
size on an interface.
The frame size on the storage system should correspond to the frame size on iSCSI Hosts (Windows and Linux
application servers).
For more specific information about how to enable Jumbo frames, see the Storage System’s manufacturer
documentation.



14
On HP StoreVirtual 4000 Storage, jumbo frames can be set from the TCP/IP settings page within the CMC console:

Note: On the storage System, make sure to set a MTU size above 8342 bytes.

Any Storage Systems configured with frames size below 8342 bytes will result in a MTU negotiation failure with the CNA,
causing the traffic to run at the default Ethernet standard frame size (i.e. 1518 bytes).


Enabling Jumbo Frames on Network Switches

Jumbo frames must be enabled across all ports of the iSCSI dedicated VLAN or hardware infrastructure (always end-to-
end). See the switch’s manufacturer documentation.
Note: Not all switches support both Jumbo Frames and Flow Control – if you have to pick between the two, choose Flow
Control.

Enabling Jumbo Frames on Virtual Connect

Jumbo Frames are enabled by default on Virtual Connect, there is no configuration required.

Enabling Jumbo Frames on Hosts Accelerated iSCSI adapters

With Emulex FlexFabric adapters:
By default, Jumbo Frames are enabled on Emulex FlexFabric adapters whenever Accelerated iSCSI is enabled, therefore
there is no configuration required. The MTU size is auto-negotiated during the TCP connection with the iSCSI target.

Note: For optimal performance, the Max MTU size supported by iSCSI Accelerated mode with Emulex FlexFabric Adapters
is limited to 8342 bytes and cannot be modified.
Note: If 9000 bytes is needed by any specific reason then Software iSCSI must be configured instead of Accelerated
iSCSI.
Note: For Software iSCSI, it might be necessary to enable Jumbo Frames at the NIC/iSCSI initiator level.










15
Under Windows:
In order to check the MTU size that has been auto-negotiated under Windows, it is necessary to install the Emulex
OneCommand Manager (OCM) utility and when launched, to select the iSCSI target and to click on Sessions…


For more information see Emulex OneCommand Manager Application User Manual, http://www-
dl.emulex.com/support/utilities/onecommand/50/onecommand.pdf

The TCPMSS value used for this connection displayed in the ‘Connection Negotiated Login Properties’ section, shows
indirectly the MTU that has been negotiated:

When TCPMSS displays 1436, the MTU negotiated size is 1514.
When TCPMSS displays 8260, the MTU negotiated size is 8342.



16
Under VMware ESX:
MTU is automatically configured and the user has no control on this setting.
To view the configured MTU enter:
# esxcfg-hwiscsi -l vmhba


With Broadcom FlexFabric adapters:
By default, when Accelerated iSCSI is enabled, Jumbo Frames are not enabled on Broadcom FlexFabric adapters,
therefore there is configuration required.

Under Windows:
To configure the MTU size under Windows, it is necessary to install the Broadcom Advanced Control Suite (BACS) utility.
For more information see the Broadcom Advanced Control Suite (BACS) section.
 When BACS is launched, select the iSCSI port and click on the Configurations tab:




17
 By default, MTU is 1500. To enable the Jumbo MTU, double-click on 1500:

 Then enter 9000 then click Apply:

 Click Yes to the warning message:





18
Under VMware ESX:
With Broadcom FlexFabric adapters Jumbo Frames must be enabled on vSwitches and VMkernel network adapters.
To enable Jumbo frames on Accelerated iSCSI virtual switches:
 Click on the Networking tab
 Select the Virtual Switches menu
 Select the vSwitch used by the hardware iSCSI adapter and click on Edit

 Set the MTU parameter (e.g. 9000) :


To set the VMkernel MTU to match the value configured on the vSwitches:
 Select the VMkernel adapters menu
 Select the VMkernel adapter linked to the vSwitch used by the hardware iSCSI adapter and click on Edit settings


19

 Select NIC settings and set the MTU parameter (e.g. 9000) :







20
Testing Jumbo Frames:
The Jumbo frames configuration can be tested by using the PING command frequently available on iSCSI Storage
Systems or directly from the server:
 Test ping from the Storage System to the iSCSI host’s interface using 9000 bytes for MTU:


The Ping result should appear similar to:
PING 192.168.20.152 (192.168.20.152) from 192.168.20.19 : 9000(9028) bytes of data.
9008 bytes from 192.168.20.152: icmp_seq=1 ttl=255 time=0.432 ms
9008 bytes from 192.168.20.152: icmp_seq=2 ttl=255 time=0.472 ms
9008 bytes from 192.168.20.152: icmp_seq=3 ttl=255 time=0.467 ms
9008 bytes from 192.168.20.152: icmp_seq=4 ttl=255 time=0.467 ms
9008 bytes from 192.168.20.152: icmp_seq=5 ttl=255 time=0.463 ms

--- 192.168.20.152 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 3999ms


 Test ping from a Windows iSCSI host using Emulex Adapters to the Storage System using 8342 bytes for MTU:

C:\Windows\system32>ping -l 8342 192.168.20.20

Pinging 192.168.20.20 with 8342 bytes of data:
Reply from 192.168.20.20: bytes=8342 time=1ms TTL=63
Reply from 192.168.20.20: bytes=8342 time=1ms TTL=63
Reply from 192.168.20.20: bytes=8342 time=1ms TTL=63
Reply from 192.168.20.20: bytes=8342 time=1ms TTL=63

Ping statistics for 192.168.20.20:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 1ms, Maximum = 1ms, Average = 1ms


21
iSCSI multipathing solutions
The use of Multipathing solutions is highly recommended for load balancing and failover to improve iSCSI performance
and availability.
Multipathing solutions use redundant physical path components–adapters, cables, and switches–to create logical
"paths" between the server and the storage device. In the event that one or more of these components fails, causing the
path to fail, multipathing logic uses an alternate path for I/O so that applications can still access their data.
For the Operating System, this multipathing means the need of an intelligent path manager called Multipath I/O (also
known as MPIO) to log in multiple sessions and to failover if needed among multiple iSCSI Host Bus Adapters (HBAs).
MPIO is a key component to building a highly available, fault tolerant iSCSI SAN solution. MPIO technologies provide for
the following:
 I/O path redundancy for fault tolerance
 I/O path failover for high availability
 I/O load balancing for optimal performance
For Microsoft Windows OS, storage vendors usually provide a vendor-specific DSM (Device Specific Module) to optimize
multipathing using the Microsoft MPIO framework. This Vendor-specific module (DSM) for multi-path I/O must be
installed under the Operating System, consult your Storage provider web site for more information.




22
Virtual Connect network scenarios

For security and performance purposes, it is recommended that the iSCSI network be separated either logically (using
different VLANs) or physically (using different physical switches) from the ordinary data network. Isolating the iSCSI
traffic helps to improve response times, reliability and prevent bottlenecks and congestion, it also helps to address the
TCP/IP overhead and flow control issues inherent in an Ethernet network.
Another recommendation to maximize availability and optimal performance is to use an iSCSI redundant network path
from Virtual Connect (and therefore from the server) to the storage system. This enables failover mechanism in case of
path failure among multiple iSCSI HBA. The use of Multipath I/O software running under the OS (Windows, Linux and
VMware) is required to provide an automatic means of persisting I/O without disconnection.
For a more complete step by step typical scenario configuration, please refer to the Virtual Connect FlexFabric Cookbook
http://bizsupport2.austin.hp.com/bc/docs/support/SupportManual/c02616817/c02616817.pdf





23
Scenario 1: iSCSI network physically separated
In this first scenario, the iSCSI network is physically separated from the ordinary data network using a different switch
infrastructure.
PROS
This scenario is the best for performance, for better latency and is the recommended scenario. It maximizes the
bandwidth availability, here the iSCSI traffic do not have to fight for bandwidth as we have a dedicated infrastructure for
the storage traffic.
CONS
This scenario uses more switches, more VC uplinks and thus more cabling. The solution cost is increased

Figure 1: Virtual Connect scenario using a separated iSCSI network


Enclosure UID
Enclosure Interlink
PS
3
PS
2
PS
1
PS
6
PS
5
PS
4
OA1 OA2
Remove management modules before ejecting sleeve
FAN
6
FAN
10
FAN
1
FAN
5
2 1
3
5
7
4
6
8
UID
X1
HP VC Flex-10 Enet Module X1 X2 X3 X4 X5 X6
SHARED
X7 X8
SHARED: UPLINK or X-LINK
UID
X1
HP VC Flex-10 Enet Module X1 X2 X3 X4 X5 X6
SHARED
X7 X8
SHARED: UPLINK or X-LINK
HP 4Gb VC-FC Module
UID
1 2 3 4
HP 4Gb VC-FC Module
UID
1 2 3 4
iLO UID
Active Reset iLO UID
Active Reset
802.3ad LAG
802.1Q Trunk
LAN Switch 1 LAN Switch 2
PRODUCTION NETWORK IP STORAGE NETWORK
iSCSI network
Rear view
c7000 enclosure
VC Modules
PRODUCTION NETWORK
802.3ad LAG
802.1Q Trunk
vNet-iSCSI-1 vNet-iSCSI-2

iSCSI Storage Device
Port 1 Port 2
Active
Virtual IP
Active
Prod-vNet-1 Prod-vNet-2
Active Active




24
Figure 2: Virtual Connect logical view of an iSCSI ESX host
HP VC Fl exFabr i c 10Gb/ 24- Por t Modul e
SHARED: UPL I NK o r X- L I NK
X3 X4 X1 X2 X5 X6 X7 X8
UID
LOM 1 LOM 2
Console
Hypervisor Host
VM Guest VLAN
1Gb
1.5Gb
3.5Gb
4Gb
1Gb
1.5Gb
3.5Gb
4Gb
A
B
C
D
A
B
C
D
VMotion
10Gb 10Gb
vmnic0
vmnic2
vmnic1
vmnic3
vmnic4 vmnic5
iSCSI2 iSCSI3
VLAN
UT
103
104
HP VC Fl exFabr i c 10Gb/ 24- Por t Modul e
SHARED: UPL I NK o r X- L I NK
X3 X4 X1 X2 X5 X6 X7 X8
UID
Prod-vNet-1
Enc0:Bay1:X1,X2
IP Storage Network
Switch 1
IP Storage Network
Switch 2
Prod-vNet-1 Prod-vNet-2
VC FlexFabric 10Gb Enc0:Bay1 VC FlexFabric 10Gb Enc0:Bay2
Virtual Connect Domain
NIC
1
NIC
2
UID
HP ProLiant
BL460cG6
serial scsi
1 port 10k
146 GB
serial scsi
1 port 10k
146 GB
UT
102
Prod-vNet-2
Enc0:Bay2:X1,X2
iSCSI Network UT
LOM 2
2B FlexiSCSI
2C FlexNIC
2D FlexNIC
2A FlexNIC
LOM 1
1C FlexNIC
1D FlexNIC
1A FlexNIC
1B FlexiSCSI
VLAN_101-1 VLAN_102-1
VLAN_103-1 VLAN_104-1
vNet-iSCSI-1
Enc0:Bay1:X5
VLAN_101-2 VLAN_102-2
VLAN_103-2 VLAN_104-2
vNet-iSCSI-2
Enc0:Bay2:X5
vSwitch



Figure 3: Virtual Connect logical view of an iSCSI Windows host
LOM 1 LOM 2
Management
Windows Host
1Gb
4Gb
5Gb
1Gb
4Gb
3.5Gb
5Gb
A
B
C
D
A
B
C
D
Application
10Gb 10Gb
LAC0
LAC2
LAC1
LAC3
LAC4 LAC5
iSCSI1 iSCSI2
VLAN
UT
HP VC Fl exFabr i c 10Gb/ 24- Por t Modul e
SHARED: UPL I NK o r X- L I NK
X3 X4 X1 X2 X5 X6 X7 X8
UID
HP VC Fl exFabr i c 10Gb/ 24- Por t Modul e
SHARED: UPL I NK o r X- L I NK
X3 X4 X1 X2 X5 X6 X7 X8
UID
iSCSI device
Port 1
iSCSI device
Port 2
Prod-vNet-1
Prod-vNet-2
VC FlexFabric 10Gb Enc0:Bay1 VC FlexFabric 10Gb Enc0:Bay2
Virtual Connect Domain
NIC
1
NIC
2
UID
HP ProLiant
BL460cG6
serial scsi
1 port 10k
146 GB
serial scsi
1 port 10k
146 GB
T
102
103
104
Prod-vNet-2
Enc0:Bay2:X1,X2
iSCSI Network UT
Team
Management-2 VLAN_102-2
VLAN_103-2 VLAN_104-2
vNet-iSCSI-1
Enc0:Bay1:X5
vNet-iSCSI-2
Enc0:Bay2:X5
Prod-vNet-1
Enc0:Bay1:X1,X2
Management-1 VLAN_102-1
VLAN_103-1 VLAN_104-1
LOM 2
2B FlexiSCSI
2C FlexNIC
2D FlexNIC
2A FlexNIC
LOM 1
1C FlexNIC
1D FlexNIC
1A FlexNIC
1B FlexiSCSI





25
Defining two iSCSI networks vNet
Create a vNet and name it “vNet-iSCSI-1”
 On the Virtual Connect Manager screen, click Define, Ethernet Network to create a vNet
 Ether the Network Name of “vNet-iSCSI-1”
 Select Smart Link, but, do NOT select any of the other options (i.e.; Private Networks etc.)
 Select Add Port, then add one port from Bay 1
 For Connection Mode, use Failover
 Select Apply






26
Create a vNet and name it “vNet-iSCSI-2”
 On the Virtual Connect Manager screen, click Define, Ethernet Network to create a vNet
 Ether the Network Name of “vNet-iSCSI-2”
 Select Smart Link, but, do NOT select any of the other options (ie; Private Networks etc.)
 Select Add Port, then add one port from Bay 2
 For Connection Mode, use Failover
 Select Apply





Note: By creating two vNets we have provided a redundant path to the network. As each uplink originates from a
different VC module and vNet both, uplinks will be active. This configuration provides the ability to lose an uplink cable,
network switch or depending on how the iSCSI ports are configured at the server (iSCSI Software Initiator supporting
failover), even a VC module.
Smart Link – In this configuration Smartlink SHOULD be enabled. Smartlink is used to turn off downlink ports within
Virtual Connect if ALL available uplinks to a vNet or SUS are down. In this scenario if an upstream switch or all cables to a
vNet were to fail on a specific vNet, VC would turn off the downlink ports connect to that vNet, which would then force
the iSCSI Software Initiator to fail-over to the alternate NIC.
Connection Mode: Failover – should be enabled here as only a single external uplink port is used for this network. With
multiple uplink ports, the connection mode Auto can be used to enable the uplinks to attempt to form aggregation
groups using IEEE 802.3ad link aggregation control protocol. Aggregation groups require multiple ports from a single
VC-Enet module to be connected to a single external switch that supports automatic formation of LACP aggregation
groups, or multiple external switches that utilize distributed link aggregation.


27
Scenario 2: iSCSI network logically separated
In this second scenario, we use the same switch infrastructure but the iSCSI network is logically separated from the
ordinary data network through the use of 802.1Q VLAN trunking.
Each Virtual Connect module is connected with more than one cable to the LAN Switches to increase the network
bandwidth and to provide a better redundancy.
PROS
This scenario uses less VC uplinks and thus less cabling. The solution cost is reduced.
CONS
In this scenario, the performance of iSCSI relies on the datacenter network performance.
If the datacenter network is congested and saturated, iSCSI performance suffers and might not be adequate for some
environments.

Figure 4: Virtual Connect scenario using a logically separated iSCSI network
Enclosure UID
Enclosure Interlink
PS
3
PS
2
PS
1
PS
6
PS
5
PS
4
OA1 OA2
Remove management modules before ejecting sleeve
FAN
6
FAN
10
FAN
1
FAN
5
2 1
3
5
7
4
6
8
UID
X1
HP VC Flex-10 Enet Module X1 X2 X3 X4 X5 X6
SHARED
X7 X8
SHARED: UPLINK or X-LINK
UID
X1
HP VC Flex-10 Enet Module X1 X2 X3 X4 X5 X6
SHARED
X7 X8
SHARED: UPLINK or X-LINK
HP 4Gb VC-FC Module
UID
1 2 3 4
HP 4Gb VC-FC Module
UID
1 2 3 4
iLO
UID
Active Reset iLO
UID
Active Reset
802.3ad LAG
802.1Q Trunk
LAN Switch 1 LAN Switch 2
iSCSI Storage Device
802.3ad LAG
802.1Q Trunk
iSCSI network
+
Production network
Rear view
c7000 enclosure
VC Modules
Port 1 Port 2
Virtual IP
PRODUCTION
NETWORK
IP STORAGE NETWORK
PRODUCTION
NETWORK
UplinkSet_1
Active
UplinkSet_2
Active
iSCSI network
+
Production network





28
Figure 5: Virtual Connect logical view of an iSCSI ESX host

HP VC Fl exFabr i c 10Gb/ 24- Por t Modul e
SHARED: UPL I NK o r X- L I NK
X3 X4 X1 X2 X5 X6 X7 X8
UID
LOM 1 LOM 2
Console
Hypervisor Host
VM Guest VLAN
1Gb
1.5Gb
3.5Gb
4Gb
1Gb
1.5Gb
3.5Gb
4Gb
A
B
C
D
A
B
C
D
VMotion
10Gb 10Gb
vmnic0
vmnic2
vmnic1
vmnic3
vmnic4 vmnic5
iSCSI2 iSCSI3
VLAN
UT
101
103
104
HP VC Fl exFabr i c 10Gb/ 24- Por t Modul e
SHARED: UPL I NK o r X- L I NK
X3 X4 X1 X2 X5 X6 X7 X8
UID
UplinkSet_1
Enc0:Bay1:X1,X2
Prod 802.1Q Trunk
(VLANs 101 through 105)
VC FlexFabric 10Gb Enc0:Bay1 VC FlexFabric 10Gb Enc0:Bay2
Virtual Connect Domain
NIC
1
NIC
2
UID
HP ProLiant
BL460cG6
serial scsi
1 port 10k
146 GB
serial scsi
1 port 10k
146 GB
UT
102
UplinkSet_2
Enc0:Bay2:X1,X2
Prod 802.1Q Trunk
(VLANs 101 through 105)
iSCSI Network UT
105
LOM 2
2B FlexiSCSI
2C FlexNIC
2D FlexNIC
2A FlexNIC
LOM 1
1C FlexNIC
1D FlexNIC
1A FlexNIC
1B FlexiSCSI
VLAN_101-1 VLAN_102-1
VLAN_103-1
iSCSI_1
VLAN_104-1
VLAN_101-2 VLAN_102-2
VLAN_103-2
iSCSI_2
VLAN_104-2
Internal Stacking link
vSwitch






29
Defining a first Shared Uplink Set (VLAN-trunk-1)
Create a SUS named “UplinkSet_1”
 On the Virtual Connect Home page, select Define, Shared Uplink Set
 Insert Uplink Set Name as UplinkSet_1
 Select Add Port, then add two ports from Bay 1
 Add Networks as follows (to add a network, right click on the grey bar under the “Associate Networks (VLAN)” header,
the select ADD;
 VLAN_101-1 = VLAN ID = 101 = CONSOLE
 VLAN_102-1 = VLAN ID = 102 = VMOTION
 VLAN_103-1 = VLAN ID = 103 = First VM Guest VLAN
 VLAN_104-1 = VLAN ID = 104 = Second VM Guest VLAN
(More VM Guest VLANs can be defined here)
 iSCSI_1 = VLAN ID=105
 Enable SmartLink on ALL networks
 Leave Connection Mode as Auto (this will create a LCAP port channel if the upstream switch is properly configured)
 Optionally, if one of the VLANs is configured as Default/untagged, on that VLAN only, set Native to Enabled
 Click Apply







30
Defining a second Shared Uplink Set (VLAN-trunk-2)
Create a SUS named “UplinkSet_2”
 On the Virtual Connect Home page, select Define, Shared Uplink Set
 Insert Uplink Set Name as UplinkSet_2
 Select Add Port, then add two ports from Bay 2
 Add Networks as follows (to add a network, right click on the grey bar under the “Associate Networks (VLAN)” header,
the select ADD;
 VLAN_101-2 = VLAN ID = 101 = CONSOLE
 VLAN_102-2 = VLAN ID = 102 = VMOTION
 VLAN_103-2 = VLAN ID = 103 = First VM Guest VLAN
 VLAN_104-2 = VLAN ID = 104 = Second VM Guest VLAN
(More VM Guest VLANs can be defined here)
 iSCSI_2 = VLAN ID=105
 Enable SmartLink on ALL networks
 Leave Connection Mode as Auto (this will create a LCAP port channel if the upstream switch is properly configured)
 Optionally, if one of the VLANs is configured as Default/untagged, on that VLAN only, set Native to Enabled
 Click Apply






31
Scenario 3: Direct-attached iSCSI Storage System
In this third scenario, an iSCSI device is directly connected to the Virtual Connect Domain without any switch
infrastructure. This scenario uses more VC uplinks than the second scenario but no additional or dedicated switches are
required like in scenario 1 to only connect a SCSI disk storage enclosure. This reduces the entire solution cost and
complexity.
PROS
Cost is greatly reduced as no additional switch is required.
CONS
There are several limitations.

Direct-attached Limitations
There are three important limitations that administrator must be aware of:
 When an iSCSI storage device is directly connected to a VC Domain, this iSCSI device is ONLY accessible to the servers
belonging to this Virtual Connect Domain
 iSCSI Storage Systems that are sharing the same ports for both iSCSI host connectivity traffic and LAN management
(also known as in-band management) can ONLY be managed from the Virtual Connect Domain
 The only network interface bond supported on the iSCSI Storage system is Active-Passive
 VC Active/Standby iSCSI vNet configuration is not supported

iSCSI Storage systems can be divided into two categories, the ones with out-of-band management (separate ports are
used for management and host traffic) and the ones with in-band management (use the same ports for management
and host traffic). So the direct-attached scenario will be divided into two sub-scenarios, according to the type of
management you may use:
 Scenario 3-A: Direct-attached iSCSI device with out-of-band management (using separate ports for management and
host traffic)
 Scenario 3-B: Direct-attached iSCSI device with in-band management (using the same ports for management and host
traffic)
 Scenario 3-C: “Low cost” Direct-attached iSCSI device with a c3000 enclosure














32
Scenario 3-A: Direct-attached iSCSI device with out-of-band management (using separate ports for management
and host traffic)
Figure 6: Virtual Connect direct-attached iSCSI target scenario with out-of-band management

iSCSI Hosts
802.3ad LAG
802.1Q Trunk
LAN Switch 1 LAN Switch 2
PRODUCTION NETWORK
Directly attached
iSCSI Storage Devi ce
Port 1
Active
Port 2
Passive
IP STORAGE NETWORK
iSCSI host network
Rear view
c7000 enclosure
VC Modules
PRODUCTION NETWORK
802.3ad LAG
802.1Q Trunk
Management network
Storage
Management
Console
vNet-iSCSI-1 vNet-iSCSI-2
Active
Virtual IP
Active
Mgt
Port
In this scenario A, the iSCSI target is connected directly to the VC Domain using two Active/Active vNet (blue and red in
the above diagram) to provide iSCSI LAN access to the servers located inside the enclosure (only).
The iSCSI storage device is using an out-of-band management network; this means that dedicated port(s) separated
from the iSCSI host traffic, are available for the management/configuration. Therefore the iSCSI device can be managed
from anywhere on the network.
Both software and hardware iSCSI can be implemented on the iSCSI hosts without limitation.






33
Note: The direct-attached scenario supports only one iSCSI device per VC Active/Active network. To support more direct-
attached iSCSI devices, it’s mandatory to create for each iSCSI device an Active/Active VC network.

LAN Switch LAN Switch

iSCSI
Target 1
Direct-attached i SCSI devices

iSCSI
Target 2
vNet1-iSCSI-1
vNet1-iSCSI-2
vNet2-iSCSI-1
vNet2-iSCSI-2
Rear view
c7000 enclosure
VC Modules
Production Network

This limitation of one device per VC Active/Active network implies that you cannot directly attach multiple HP
StoreVirtual 4000 Storage nodes to the same VC domain because in a multi node environment, StoreVirtual nodes need
to talk to each other and this requirement is not possible as VC don’t switch traffic between different VC networks.


LAN Switch LAN Switch

HP P4000 series
Node 1
Direct-attached HP LeftHand P4000 series
Rear view
c7000 enclosure
VC Modules
Production Network

HP P4000 series
Node 2






34
Scenario 3-B: Direct-attached iSCSI device with in-band management (using the same ports for management and
host traffic):
Figure 7: Virtual Connect direct-attached iSCSI target scenario with in-band management


802.3ad LAG
802.1Q Trunk
LAN Switch 1 LAN Switch 2
PRODUCTION NETWORK IP STORAGE NETWORK
iSCSI host network +
management network
Rear view
c7000 enclosure
VC Modules
PRODUCTION NETWORK
802.3ad LAG
802.1Q Trunk
Storage
Management
Console
iSCSI Hosts
vNet-iSCSI-1 vNet-iSCSI-2
Directly attached
iSCSI Storage Devi ce
Port 1
Active
Port 2
Passive
Active
Virtual IP
Active

In this scenario B, the iSCSI device is again connected directly to the VC Domain using two Active/Active vNet (blue and
red in the above diagram) but here the iSCSI device is not using a dedicated interface for management which means that
the same ports are used for both the management and the iSCSI host traffic (known as in-band management device).
This implies due to the Virtual Connect technologies that you can only manage and configure the iSCSI storage device
from the Virtual Connect Domain (i.e. only from a server located inside the enclosure and connected to the iSCSI vNet).
This means that you need a dedicated server for the storage system management and that all SNMP trap notifications
for hardware diagnostics, events and alarms will be sent only to this management console.
Both software and hardware iSCSI can be implemented here without limitation on the iSCSI hosts.
For more information about Accelerated iSCSI, please refer to the Accelerated iSCSI section.





35
Scenario 3-C: “Low cost” Direct-attached iSCSI device with or without out-of-band management
In this scenario, the iSCSI Storage device is directly connected to a c3000 enclosure filled with only one Virtual Connect
module installed into interconnect Bay 1. By design, interconnect Bay 1 in c3000 enclosures always map to all embedded
NICs/CNAs, so all embedded server ports are all connected through the same interconnect module.
This scenario can be proposed to customers looking for cost reduction and not necessarily for high availability.
PROS
Cost is even more greatly reduced as only one VC module is required.
CONS
Risk of network failure. In addition to the direct-attached limitations described before, this scenario does not offer a
fully redundant design: the VC module is a single point of failure, if it fails, the entire network communication will stop
and all blades will go offline.


Figure 8: Virtual Connect direct-attached iSCSI target scenario using a single VC module


PS5
PS4 FAN6
PS6
FAN3 FAN1
PS3
PS2
PS1 FAN4
OA1/iLO Enc.
Int er link Encl osure
UID OA2/iLO
iSCSI Hosts
802.1Q Trunk
LAN Switch 1 LAN Switch 2
PRODUCTION NETWORK
Directly attached
iSCSI Storage Devi ce
Port 1
Active
Port 2
Passive
IP STORAGE NETWORK
Rear view
c3000 enclosure
filled with only one VC module
VC Flex-10/10D Module
in Bay 1
PRODUCTION NETWORK
802.1Q Trunk
vNet-iSCSI-1 vNet-iSCSI-2
Active
Virtual IP
Active
Optional
out-of-band
management
Mgt
Port




36
Virtual Connect Network configuration

Scenario 3-A and Scenario 3-B

The Virtual Connect network configuration is the same for Scenario 3-A and Scenario 3-B.
You must define two Active/Active iSCSI Virtual Connect networks (vNet): “vNet-iSCSI-1” and “vNet-iSCSI-2” like in
Scenario 1.
 These two vNets can exclusively be used for the direct attachment of the iSCSI storage device
 Each vNet can only support one VC uplink
 Additional vNets are required for the non-iSCSI traffic

Scenario 3-C

You must define two Active/Active iSCSI Virtual Connect networks (vNet): “vNet-iSCSI-1” and “vNet-iSCSI-2” like in
Scenario 1. The only difference being that uplink ports from the same VC Module in Bay 1 are used for both vNets.
 These two vNets can exclusively be used for the direct attachment of the iSCSI storage device
 Each vNet can only support one VC uplink
 Each vNet is configured with one VC uplink port from the module Bay 1
 Additional vNets are required for the non-iSCSI traffic


Note: The Direct-attached scenario is not supported with VC Active/Standby iSCSI vNet configuration. A standby VC
uplink is always seen as active by an upstream device so having no way to detect the standby state of the VC uplink, a
storage system would incorrectly send traffic to the standby port, causing communication issues.




37
Connecting the Direct-Attached iSCSI SAN to the VC Domain
Connect the VC uplink port of the two vNet to the network interfaces of the iSCSI device.

Figure 9: Out-of-band management configuration with a HP StoreVirtual 4000 Storage Solution (Scenario 3-A)
Mgmt
2 1 UID
P O R T 1
1 0 G b E
S F P
P O R T 2
L A
L A
T X R X T X R X
NC550sp
HP VC Fl exFabr i c 10Gb/ 24- Por t Modul e
SHARED: UPL I NK o r X- L I NK
X3 X4 X1 X2 X5 X6 X7 X8
UID
HP VC Fl exFabr i c 10Gb/ 24- Por t Modul e
SHARED: UPL I NK o r X- L I NK
X3 X4 X1 X2 X5 X6 X7 X8
UID
NIC 1
NIC 2
UID
HP ProLiant BL460c
CNA 1 CNA 2
Prod-vNet-1 Prod-vNet-2
HP StoreVirtual 4000 Storage
NIC 1
NIC 2
UID
HP ProLiant BL460c
CNA 1 CNA 2 NIC 1
NIC 2
UID
HP ProLiant BL460c
CNA 1 CNA 2 NIC 1
NIC 2
UID
HP ProLiant BL460c
CNA 1 CNA 2
iSCSI Hosts
Blade Servers
vNet-iSCSI-1 vNet-iSCSI-2
Active Active
Directly attached iSCSI device
using out-of-band management
HP VC FlexFabric
Management network

Figure 10: In-band management configuration with a HP StoreVirtual 4000 Storage Solution (Scenario 3-B)
Mgmt
2 1 UID
HP VC Fl exFabr i c 10Gb/ 24- Por t Modul e
SHARED: UPL I NK o r X- L I NK
X3 X4 X1 X2 X5 X6 X7 X8
UID
HP VC Fl exFabr i c 10Gb/ 24- Por t Modul e
SHARED: UPL I NK o r X- L I NK
X3 X4 X1 X2 X5 X6 X7 X8
UID
NIC 1
NIC 2
UID
HP ProLiant BL460c
CNA 1 CNA 2
Prod-vNet-1 Prod-vNet-2
HP StoreVirtual 4000 Storage
NIC 1
NIC 2
UID
HP ProLiant BL460c
CNA 1 CNA 2 NIC 1
NIC 2
UID
HP ProLiant BL460c
CNA 1 CNA 2 NIC 1
NIC 2
UID
HP ProLiant BL460c
CNA 1 CNA 2
iSCSI Hosts
Blade Servers
vNet-iSCSI-1 vNet-iSCSI-2
Active Active
Directly attached iSCSI device
using in-band management
HP VC FlexFabric




38
Preparing the network settings of the storage system
The following describes a basic network configuration of a HP StoreVirtual 4000 Storage Solution:
 Open a remote console on the StoreVirtual using the StoreVirtual iLO or connect a keyboard/monitor
 Type start, and press Enter at the log in prompt
 Press Enter to log in or type the Username and password if already configured
 When the session is connected to the storage system, the Configuration Interface window opens
 On the Configuration Interface main menu, tab to Network TCP/IP Settings, and press Enter
 Tab to select the first network interface and press Enter
 Enter the host name, and tab to the next section to configure the network settings
 Enter a Private IP address like 192.168.5.20 / 255.255.255.0 with 0.0.0.0 for the Gateway address:

 Tab to OK, and press Enter to complete the network configuration
 Press Enter on the confirmation window. A window opens listing the assigned IP address that will be used later on to
configure and manage the P4300


Configuring the Storage Management Server
Out-of-band management iSCSI device (Scenario 3-A and 3-C)
An iSCSI device with an out-of-band management does not require any particular configuration as the management
console can reside anywhere on the network.
In-band management iSCSI device (Scenario 3-B and 3-C)
An iSCSI device with an in-band management requires a little bit more attention and particularly when network interface
bonding is enabled on the iSCSI Storage System.
The following configuration can be followed for an in-band management iSCSI Storage System:
 Create a Virtual Connect server profile for the Management Storage server:
 Assign NIC1 and NIC2 to the management network
 Assign NIC3 and NIC4 to the iSCSI direct-attached VC networks “vNet-iSCSI-1” and “vNet-iSCSI-2”



39
Figure 11: Example of a VC profile used by the Storage Management Server

 Start the server and install Windows Server 2003 or 2008
 At the end of the installation, assign a private IP address to NIC3 (e.g. 192.168.5.150) and to NIC4 (e.g. 192.168.5.151)

Note: Make sure to use here the same IP subnet addresses as the one set in the Storage System!

Note: Two IP addresses are used in this configuration to ensure that the management server stays always connected
to the iSCSI device regardless of the iSCSI device bonding configuration (no bonding or Active-Passive bond enabled).
Despite the use of the Active/Active iSCSI vNet configuration, NIC teaming cannot be used on the management server
otherwise the network connection to the iSCSI device will fail. This is due to the direct-attached design and the use of
NIC bonding on the iSCSI device. There is no such limitation with Scenario 3-A.




40
 Install the Storage Management software (e.g. HP StoreVirtual Management Software, this is the software used to
configure and manage HP StoreVirtual Storage)

 Open the Centralized Management Console (CMC) and locate the storage system using the Find function, enter the
address assigned previously on the first interface of the P4300 (e.g. 192.168.5.20)

 Select the Auto Discover by Broadcast check box
 When you have found the storage system, it appears in the Available systems pool in the navigation window

Configuring the network interface bonds on the Storage System
When configuring an iSCSI SAN directly attached to VC, one point is worth considering: the network interface bonding.
Network interface bonding is generally used on iSCSI SAN devices to provide high availability, fault tolerance, load
balancing and/or bandwidth aggregation. Depending on your storage system hardware, you can generally bond NICs in
one of the following methods:
 Active – Passive
 Link Aggregation (802.3 ad)
 Adaptive Load Balancing
Related to the direct-attached scenario, when a bond is configured, multiple interfaces are used but due to lack of
Ethernet switches between the VC domain and the iSCSI SAN device, the support of NIC bonding is very limited: only the
Active/Passive bonding method is supported.

Note: Link Aggregation (802.3ad) is not supported because you cannot create an aggregation group across different VC
modules. Adaptive Load Balancing is not supported either as it requires the two interfaces to be connected to the same
network which is not the case here as we have two vNet for iSCSI (Active/Active scenario).
To configure Active/Passive bonding on a HP StoreVirtual 4000 Storage Solution:


41
 From CMC, select the storage system, and open the tree below it
 Select TCP/IP Network category, and click the TCP/IP tab

 Select the two interfaces from the list for which you want to configure the bonding, right-click and select New Bond…
 Select the Active – Passive bond type

 Click OK
 After a warning message, click OK to rediscover the Storage system, the IP address is normally preserved
 After a few seconds, the new bond setting shows the Active-Passive configuration:

 The Storage System is now ready to be used with fault tolerance enabled






42
Configuring the iSCSI Host
For the iSCSI Hosts configuration, Hardware or/and Software iSCSI connection can be used.
Multipathing and path checking
With the Direct-attached scenario and its Active – Passive iSCSI SAN NIC bonding, it is recommended to check the second
path to validate the entire configuration. After the iSCSI volume has been discovered from the first path, log in to the
iSCSI Storage System and trigger a NIC bonding fail-over, this will activate the second interface, you will then have the
ability to validate the second iSCSI path.





43
Accelerated iSCSI

Traditional software-based iSCSI initiators generate more processing overhead for the server CPU. An accelerated iSCSI
capable card, also known as Hardware iSCSI, “offloads” the TCP/IP operations from the server processor freeing up CPU
cycles for the main applications.
The main benefits are:
 Processing work offloaded to the NIC to free CPU cores for data-intensive workloads
 Increased server and IP storage application performance
 Increased iSCSI performance
Software-based iSCSI initiator is supported by all Virtual Connect modules but Accelerated iSCSI (Hardware-based iSCSI
initiator) is only available with the component listed in the System Requirement section.

Note: Under the OS, Accelerated iSCSI defers with Software iSCSI in the sense that it provides a HBA type of interface
and not a Network interface card (NIC). Consequently, additional drivers, software and settings are sometimes required.
Note: The selection between Software iSCSI and Accelerated iSCSI is done under the Virtual Connect profile:
 Example of a server profile with Software iSCSI enabled:



44
 Example of a server profile with Accelerated iSCSI enabled:



Several steps are required to connect a server to an iSCSI target using accelerated iSCSI:
 Accelerated iSCSI must be enabled on the server using Virtual Connect
 A hardware iSCSI initiator must be installed and configured under the OS
 A Multipath IO software must be used to manage the redundant iSCSI connection
This document will provide the steps to enable Accelerated iSCSI on a BladeSystem server under Microsoft Windows
Server and VMware vSphere Server.







45
Creating a Virtual Connect profile with Accelerated iSCSI
The first step is the same for both MS Windows and VMware vSphere servers; we have to create a Virtual Connect server
profile that enables Accelerated iSCSI:
 Open Virtual Connect manager. From the Menu Define, select Server Profile in order to create a new VC profile

Note that VCM assigns FCoE connections by default!

Note: A server with one FlexFabric adapter can be configured with a unique personality, either all Ethernet, or
Ethernet/iSCSI, or Ethernet/FCoE. Therefore it’s not possible to enable at the same time both FCoE and iSCSI
connections. Only a server with multiple FlexFabric Adapters can be configured with both iSCSI and FCoE connections.

 If you have a unique FlexFabric Adapter, delete the two FCOE connections, otherwise jump to the next bullet

 In the iSCSI HBA Connections section, click Add



46
 In the Network Name column, click on Select a network…

 Select your iSCSI dedicated VC network and click OK

 In the Port Speed column, you can adjust the speed settings (Auto – Preferred – Custom)

 In the Boot setting column, leave DISABLED


Note: Boot setting disabled means Accelerated iSCSI is enabled but iSCSI Boot is unavailable.
The disable mode offloads the iSCSI protocol processing from the OS to the NIC. In addition to offloading TCP/IP protocol
processing, it also offloads iSCSI protocol processing.
Note: The multiple networks feature (i.e. when using 802.1Q VLAN tagging) is not supported for iSCSI HBA connections.




47
 Optionally create a second iSCSI Connection for multipathing configuration


Note: Allowing more than one iSCSI application server to connect to a volume concurrently without cluster-aware
applications or without an iSCSI initiator with Multipath I/O software could result in data corruption.

 Configure the additional VC Ethernet Network connections that may be needed on the other FlexNIC:

 When done, you can assign the profile to a server with an Ethernet adapter that supports Accelerated iSCSI:

 Click Apply to save the profile

 The server can now be powered on (using either the OA, the iLO, or the Power button)


48
Accelerated iSCSI with Microsoft Windows Server
After the creation of a VC profile with Accelerated iSCSI enabled, we need to proceed with the following steps under
Microsoft Windows Server:
 Installation of the vendor’s FlexFabric adapter utility (i.e. Emulex OneCommand Manager or Broadcom Advanced
Control Suite)
 IP Configuration of the iSCSI HBA ports
 Enabling Jumbo Frames and Flow Control on the NIC adapters
 Installation of the Microsoft iSCSI initiator
 Connecting volumes with the Microsoft iSCSI Initiator
 Installation of Microsoft MPIO
 Claim an iSCSI-attached device for use with MPIO
 Installation of the Device Specific Module (DSM)
 Using the Microsoft DSM that comes with Microsoft MPIO
 Using the Storage vendor DSM for MPIO provided by the Storage System vendor for better performance and
latency


Installation of the vendor’s adapter utility

Installing Emulex OneCommand Manager
OneCommand Manager is the Emulex utility to manage the NC55x FlexFabric Converged Network Adapters. Among other
things, it provides comprehensive control of the iSCSI network including discovery, reporting and settings.
For accelerated iSCSI under MS Windows server, the Emulex OneCommand Manager is mandatory as it provides the only
way to configure the IP settings of the iSCSI HBA ports required to connect the iSCSI volume.
For more information about OneCommand Manager, refer to:
 The Emulex OneCommand Manager User Manual and Release Notes:
http://www.emulex.com/products/software-solutions/onecommand-manager/downloads/ (select any Windows
OS)
 The OneCommand Manager section further below in this document

Note: Do not confuse between hardware and software iSCSI initiators. Accelerated iSCSI always uses a specific port (i.e.
host bus adapters) and require a utility from the HBA vendor (i.e. Emulex OneConnect Manager, Emulex Drivers be2iscsi).
Storage iSCSI
Adapter Ports
Network Interface Ports

On the opposite, software initiators use the standard server NIC port and are usually included in the Operating System
(i.e. Microsoft iSCSI initiator).


49
To download OneCommand Manager, go to the Emulex web site (you must register):
http://www.emulex.com/products/software-solutions/onecommand-manager/downloads/
 Select the Windows version:

 Select the Enterprise Kit:


Note: The HP OneCommand Enterprise Kit contains a graphical User Interface (GUI) and a Command Line Interface
(CLI). HbaCmd.exe (the CLI) is located by default in C:\Program Files\Emulex\Util\OCManager




50
 Launch the OneCommand utility:
 For Windows 2012: press the Windows logo key to open the Start screen and click on OCManager

 For Windows 2008: from the start menu, select All Programs / Emulex / OCManager



51
 OneCommand Manager shows all iSCSI ports detected on the server:


Installing Broadcom Advanced Control Suite
Broadcom Advanced Control Suite is the Broadcom utility to manage the FlexFabric 53x and 63x Converged Network
Adapters. Among other things, it provides comprehensive control of the iSCSI network including discovery, reporting and
settings.
For accelerated iSCSI under MS Windows server, the Broadcom Advanced Control Suite is mandatory as it provides the
only way to configure the IP settings of the iSCSI HBA ports required to connect the iSCSI volume.
For more information about Broadcom Advanced Control Suite and Broadcom Adapters, refer to:
 Broadcom Advanced Control Suite web page :
http://www.broadcom.com/support/ethernet_nic/management_applications.php
 The Broadcom Advanced Control Suite section further below in this document

To download the Broadcom Advanced Control Suite (BACS), go to the Broadcom web site:
http://www.broadcom.com/support/ethernet_nic/netxtremeii10.php#mgmt_apps

 To launch the Broadcom Advanced Control Suite:
 For Windows 2012: press the Windows logo key to open the Start screen and click on Broadcom
Advanced…

 For Windows 2008: in Control Panel, click Broadcom Control Suite 4




52
 Broadcom Advanced Control Suite shows all iSCSI ports detected on the server:




IP Configuration of the iSCSI HBA ports

Configuring the IP address of an Emulex adapter’s iSCSI port
In order to log into the iSCSI target, an IP address must be assigned to each iSCSI HBA ports:
 From the OneCommand Manager interface, select the first iSCSI port, and click Modify




53
 Enter a static IP address or check DHCP Enabled

This IP address must be in the same subnet as the one configured in the iSCSI Storage System.

Note: The multiple networks feature being not supported for iSCSI HBA connections (see Creating a Virtual Connect
profile with Accelerated iSCSI) the ‘VLAN Enabled’ option must remain unchecked for all scenarios presented in this
cookbook (i.e. scenario 1, scenario 2 and scenario 3).
The ‘VLAN Enabled’ option can be required when Virtual Connect is tunneling the iSCSI traffic (i.e. when the iSCSI
network under VCM has the ‘Enable VLAN Tunneling’ checkbox selected). The use of a Tunnel network for the iSCSI
traffic is not presented in this cookbook because the most common scenarios are using the iSCSI vNet in VLAN
mapping mode (i.e. not using the VLAN tunneling mode).

 Click OK
 Repeat the same steps for the second iSCSI port


Configuring the IP address of a Broadcom adapter’s iSCSI port
In order to log into the iSCSI target, an IP address must be assigned to each iSCSI HBA ports:
 From the Broadcom Advanced Control Suite interface, select the first iSCSI port, and click on Configurations tab:



54

 And click on Edit for the IPv4 Configuration:

 Leave DHCP Enabled or enter a static IP address:

This IP address must be in the same subnet as the one configured in the iSCSI Storage System.

 Click OK
 Repeat the same steps for the second iSCSI port




Enabling jumbo frames and Flow Control on the NIC adapters

See Enabling Flow Control on Host Accelerated iSCSI adapters and Enabling Jumbo Frames on Hosts Accelerated
iSCSI adapters sections.



55
Installation of the Microsoft iSCSI initiator
The Microsoft iSCSI Software Initiator enables connection of a Windows host to an external iSCSI storage array using
Ethernet NICs.
For more information about the Microsoft iSCSI initiator, see the Step-by-Step Guide http://technet.microsoft.com/en-
us/library/ee338476%28WS.10%29.aspx

Note: The Microsoft iSCSI Initiator utility is a software initiator (using Ethernet NICs) but it can as well be used to manage
the iSCSI accelerated connectivity (using the iSCSI HBAs).

For Windows 2012:
The Microsoft iSCSI initiator comes installed by default.
To launch the Microsoft iSCSI initiator:
 Go to Server Manager
 Then select Tools / iSCSI Initiator

 The first time the iSCSI Initiator is launched, the iSCSI service is required to be started, click Yes




56
 The following iSCSI Initiator Properties windows appears:


For Windows 2008:
The Microsoft iSCSI initiator comes installed with both the Windows Server 2008, and the Server Core installation.





57
Connecting volumes with the Microsoft iSCSI Initiator
 Open the Microsoft iSCSI Initiator:
 For Windows 2012, open the Server Manager console, go to Tools then select iSCSI Initiator


 For Windows Server 2008, open the Control Panel then enter iSCSI in the search tab





58
 From the Targets tab, enter the iSCSI Storage System IP address then click Quick Connect…

Note: iSCSI SAN vendors usually recommend using a Virtual IP address; refer to the iSCSI SAN vendor documentation.

 A window pops up showing the status of the iSCSI connection




59
 Select the volume to log on to and click Connect

 Select the Enable multi-path check box


Note: Do not select the “Enable Multi-path” checkbox if your iSCSI Storage System is not supporting load balanced iSCSI
access.

 Click OK to finish







60
Installation of Microsoft MPIO
MPIO solutions are needed to logically manage the iSCSI redundant connections and ensure that the iSCSI connection is
available at all times. MPIO provides fault tolerance against single point of failure in hardware components but can also
provide load balancing of I/O traffic, thereby improving system and application performance.
MPIO is an optional component in all versions of Windows 2012 Server so it must be installed:

For Windows 2012:

 Go to Server Manager
 Then select Manage
 Then Add Roles and Features
 In the Features section, select Multipath I/O
 Click Next and Install


For Windows 2008:

 Go to Server Manager
 Then select Features
 Then Add Features
 Select Multipath I/O
 Click Next and Install


When the installation has completed, you need to restart the computer, click Yes.
After restarting the computer, the computer finalizes the MPIO installation.

Claim an iSCSI-attached device for use with MPIO
To claim an iSCSI-attached device for use with MPIO:
 Open the MPIO control panel by clicking Start, Control Panel, point to Administrative Tools, and then click MPIO




61
 The MPIO Devices tab is displaying the hardware IDs of the devices that are managed by MPIO:

 Click the Discover Multi-Paths tab, select the Add support for iSCSI devices check box, and click Add

Note: This check box does not become active unless you have an iSCSI-attached device.


62
 The system asks to reboot for the policy to take effect, click Yes

 After the reboot, re-open the MPIO control panel, the MPIO Devices tab lists the additional hardware ID
“MSFT2005iSCSIBusType_0x9”

Note: When hardware ID “MSFT2005iSCSIBusType_0x9” is listed, all iSCSI bus attached devices will be claimed by the
Microsoft DSM.



63
Installation of the Device Specific Module (DSM)
MPIO requires the installation of device specific modules (DSM) in order to provide support for using multiple data paths
to a storage device. These modules are a server-side plug-in to the Microsoft MPIO framework.
A native Microsoft DSM provided by Microsoft comes by default with Microsoft MPIO software but storage providers can
develop their own DSM that contain the hardware specific information needed to optimize the connectivity with their
storage arrays.
Vendor specific DSM usually provides optimal performance, lower latency, more advanced path decisions and quicker
path failover times while the Microsoft DSM offers more basic functions.

Installation of the Microsoft DSM
With Windows 2003 Server, the Microsoft DSM comes with the Microsoft iSCSI Initiator.
With Windows 2008 and 2012 Server, the Microsoft DSM is an optional component that comes with MPIO. When MPIO is
installed, the Microsoft device-specific module (DSM) is also installed. For more information, see:
- Microsoft Multipath I/O (MPIO) User’s Guide for Windows Server 2012 at http://www.microsoft.com/en-
us/download/details.aspx?id=30450
- Installing and Configuring MPIO for Windows 2008 Server at http://www.microsoft.com/mpio

Installation of the Storage vendor DSM
To determine which DSM to use with your storage, refer to information from your hardware storage array manufacturer.
Many storage arrays that are SPC-3 compliant will work by using the Microsoft MPIO DSM but some manufacturers
provide their own DSMs to achieve optimal performance.
Note: Refer to your storage array manufacturer for information about which DSM to use with a given storage array, as
well as the optimal configuration of it.
 Download the Vendor’s DSM for your iSCSI storage array

Note: The HP StoreVirtual 4000 Storage DSM for MPIO software is not supported with Accelerated iSCSI HBA. For more
information see HP StoreVirtual DSM for Microsoft MPIO Deployment Guide,
http://h10032.www1.hp.com/ctg/Manual/c03983714.pdf

Note: As a recommendation, if you are not sure about your storage array DSM support with Accelerated iSCSI adapters
or if you are facing issues after the vendor’s DSM installation, it is recommended to use the Microsoft DSM that comes
with the Microsoft iSCSI Initiator.

 Install the Vendor’s DSM. A reboot might be required

Note: Once the Vendor’s DSM for MPIO is installed, there are sometimes additional parameters specific for HP
FlexFabric adapters (CNAs) so refer to your storage array manufacturer’s documentation. All iSCSI volume
connections made to an iSCSI Storage System will attempt to connect with the Storage vendor’s DSM for MPIO.



64
 After the reboot, re-open the MPIO control panel, the MPIO Devices tab lists additional hardware IDs that have been
installed:


Note: When the vendor’s hardware ID is listed, the iSCSI storage array will be claimed by the Vendor’s DSM.

 Open now the iSCSI initiator control panel, click Connect




65
 Select the Enable multi-path check box and click OK



Multipath Verification

 You can verify the DSM for MPIO operations, from the Target tab, select the volume and click on Properties:




66
 You should see here the multiple sessions of this target as the DSM for MPIO automatically builds a data path to each
storage node in the storage cluster:

If after a few minutes only one session is available (you can click on refresh several times) it could be necessary to
configure manually the multipathing.

Manual Multipath configuration

 Click on Add session
 Select the Enable multi-path check box and click Advanced




67
 Configure the Advanced Settings as follows:

 For Local adaptor, select the Emulex FlexFabric adapters:

or with Broadcom FlexFabric Adapters:

 For Source IP, select the IP address of the iSCSI HBA to connect to the volume
 For Target portal, select the IP of the iSCSI Storage System containing the volume


 Click OK to close the Advanced Settings dialog
 Click OK to finish logging on
 Repeat the same steps using this time to the second Accelerated iSCSI adapter

Note: It is recommended to test the failover to validate the multipath configuration.





68
Using the Microsoft iSCSI Software Initiator in conjunction with Accelerated iSCSI support
Besides the Accelerated iSCSI attached disk, it is possible to use as well the iSCSI Software to connect additional volumes
using one of the NIC adapter(s). This section describes the different configuration steps required.

Virtual Connect Profile Configuration

From the Windows server’s VC Profile, make sure that at least one server NIC is connected to an iSCSI network or to a
network where an iSCSI device relies:

Note: It is recommended to use a dedicated network adapter for iSCSI with a suitable bandwidth.


Software iSCSI
Accelerated iSCSI


69
Enabling jumbo frames on the NIC adapters

As an optional recommendation, you can enable Jumbo frames on the two NIC adapters connected to iSCSI-1 and iSCSI-2
VC Networks.
 Under MS Windows Server 2012:
1. In Control Panel, select View network status and tasks in the Network and Internet section
2. In the View your basic network information and set up connections section, click on the network adapter used for
iSCSI and click Properties
3. Click Configure
4. Click the Advanced tab
5. Select Packet Size (with Emulex adapters) or Jumbo Packet (with Broadcom adapters) and change the MTU value
supported by the network infrastructure and the iSCSI Storage Array:

With Emulex adapters With Broadcom adapters

 Under MS Windows Server 2008:
1. From the HP Network configuration Utility, select the port connected to the iSCSI Network
2. Click Properties
3. Select the Advanced Settings tab then click on Jumbo Packet


70
4. Select the MTU size supported by the network infrastructure and the iSCSI Storage System

5. Click OK to close






71
Adding iSCSI Software connections with multiple NICs

 Open the Microsoft iSCSI initiator
 From the Targets tab, enter the iSCSI Storage System IP address then click Quick Connect…

Note: Enter the IP Address of the iSCSI target system connected to one of the server NIC network (e.g. iSCSI-1 and
iSCSI-2)

 A window pops up showing the status of the iSCSI connection

 Click Done


72
 A new discovered iSCSI target should appear in the list if a LUN has been correctly presented to the server.


 Select the new target and click Properties

 Click Add session


73
 Select Enable multi-path then click Advanced

 Configure the Advanced Settings as follows:
 For Local adapter, select Microsoft iSCSI Initiator

 For Initiator IP, select the IP address of the first NIC connected to iSCSI-1 VC Network



74
 For Target portal, select the IP of the iSCSI Storage System containing the volume

 Click OK to close the Advanced Settings dialog
 Click OK to finish
 Repeat the same steps but this time using the second NIC adapter connected to iSCSI-2 VC Network and using the
corresponding IP address

 You should see now several sessions using the first and second paths:



75
 Click OK to close the Properties window


Two iSCSI targets are now available via the Microsoft iSCSI initiator, one is using the iSCSI Accelerated ports (iSCSI
HBAs) and the other is using the NIC adapters with software iSCSI.



76
Accelerated iSCSI with VMware vSphere Server


After the creation of a Virtual Connect server profile with Accelerated iSCSI enabled, we can proceed with the VMware
vSphere configuration however there is an important point to remember, under VMware vSphere, hardware iSCSI
adapters are divided into two categories:
 Dependent hardware iSCSI adapters (i.e. HP FlexFabric 53x/63x adapters from Broadcom)
 Independent hardware iSCSI adapters (i.e. HP FlexFabric 55x adapters from Emulex)

And for each category, we need to proceed with different installation steps described below:
 Installation of the iSCSI drivers of the FlexFabric adapter
 Installation of the vendor’s FlexFabric adapter utility (optional)
 Configuration of the hardware iSCSI adapter:
- When a Broadcom FlexFabric dependent hardware iSCSI adapter is used:
 Configuration of the network for iSCSI
- Determining the association between the dependent hardware adapters and physical NICs
 Configuration of the network port binding
 Configuration of the discovery information
 Checking multipath information
 Enabling Jumbo Frames with iSCSI (must be enabled on vSwitches and VMkernel network adapters) and
Flow Control
- When an Emulex FlexFabric independent hardware iSCSI adapter is used:
 Configuration of the IP address of the Emulex iSCSI ports
 Configuration of the discovery information
 Checking multipath information
 Enabling Jumbo Frames and Flow Control on the vmhba adapters


For more information about Accelerated iSCSI and VMware vSphere, see Best Practices for Running VMware vSphere on
iSCSI, https://www.vmware.com/files/pdf/iSCSI_design_deploy.pdf


77
Description of the two categories:
 Dependent Hardware iSCSI Adapters:
This category contains the HP FlexFabric 53x/63x adapters (from Broadcom).
These adapters depend on VMware networking and iSCSI configuration, in other words, the IP address of the system is
obtained from the host NIC connections (vmnic).
This type of adapter presents a standard network adapter (vmnic) and iSCSI offload functionality (vmhba) for the
same port.


















The iSCSI offload functionality depends on the host's network configuration to obtain the IP, MAC, and other
parameters used for iSCSI sessions.

















Unlike with independent
Hardware iSCSI Adapters,
there is no IP, MAC, and other
parameters options in the
iSCSI adapter properties.


78
With these adapters, you must create network connections to bind dependent iSCSI (vmhba) and network adapters
(vmnic). You must determine the name of the physical NIC with which the dependent hardware iSCSI adapter is
associated.


Jumbo Frames must be enabled on vSwitches and VMkernel network adapters.




79
 Independent Hardware iSCSI Adapters: This category contains the HP FlexFabric 55x adapters (from Emulex).
These adapters implements its own networking and iSCSI configuration and management interfaces, in other words,
the IP address is entered into the vmhba card.
This type of adapter presents only an iSCSI offload functionality (vmhba) and no standard network adapter (vmnic).



The iSCSI offload functionality has independent configuration management that assigns the IP, MAC, and other
parameters used for the iSCSI sessions that are found in the iSCSI Adapter Properties:

Jumbo Frames are not supported by VMware on Independent Hardware iSCSI Adapter however with Emulex FlexFabric
adapters, the MTU size is auto-negotiated during the TCP connection with the iSCSI target, and therefore there is no
configuration required.
Unlike with dependent Hardware
iSCSI Adapters, there is no vmnic
which corresponds to the iSCSI
HBA connections.


80
Installing iSCSI drivers of the FlexFabric adapter

 Verity that the adapter is installed correctly.
 Make sure that the latest FlexFabric adapter’s iSCSI drivers for VMware vSphere have been installed.
To obtain these drivers, click below on the link corresponding to the FlexFabric adapter you have in the server:

HP NC551i 10Gb 2-port FlexFabric
Adapter
http://h20565.www2.hp.com/portal/site/hpsc/public/psi/swdHome
/?cc=us&cc=us&lang=en&lang=en&sp4ts.oid=5033634&ac.admitte
d=1393861522084.876444892.492883150
HP NC553i 10Gb 2-port FlexFabric
Adapter
http://h20565.www2.hp.com/portal/site/hpsc/public/psi/swdHome
/?cc=us&cc=us&lang=en&lang=en&sp4ts.oid=4324630&ac.admitte
d=1393861493136.876444892.492883150
HP FlexFabric 10Gb 2-port 554FLB
Adapter
http://h20565.www2.hp.com/portal/site/hpsc/public/psi/swdHome
/?cc=us&cc=us&lang=en&lang=en&sp4ts.oid=5215362&ac.admitte
d=1393861408870.876444892.492883150
HP FlexFabric 10Gb 2-port 534FLB
Adapter
http://h20565.www2.hp.com/portal/site/hpsc/public/psi/swdHome
/?cc=us&cc=us&lang=en&lang=en&sp4ts.oid=5404530&ac.admitte
d=1393860649911.876444892.492883150

Then select your VMware version and download the Emulex or Broadcom iSCSI Driver.

Note: Hardware and software iSCSI can be managed by vSphere without any additional utility. Accelerated iSCSI uses a
specific port (i.e. vmhba) and requires the installation of HBA drivers included in the iSCSI package.

When the FlexFabric adapter is properly configured, vmhba iSCSI adapters are displayed on the list of Storage Adapters:
 Log in to the vSphere web Client, and select your host
 Select the Manage menu and click on the Storage tab
 Select the Storage Adapters menu. In the storage adapters view, iSCSI vmhba adapters are displayed:

With Emulex adapters


81

With Broadcom adapters

Note: If it is installed, the hardware iSCSI adapter appears on the list of storage adapters under such category as, for
example, Broadcom iSCSI Adapter.

Installing the vendor’s FlexFabric Adapter utility (optional)
Each vendor of the FlexFabric adapters provides a management utility for their adapters:
 OneCommand Manager (OCM) is the Emulex utility to manage the Emulex 55x FlexFabric adapters
 Broadcom Advanced Control Suite (BACS) is the Broadcom utility but BACS is not available at this time for VMware
vSphere.

OCM is not required to connect a VMware server to an iSCSI volume but it delivers lots of advanced management
features, configuration, status monitoring and online maintenance.
To install OneCommand Manager under VMware, refer to the OneCommand Manager for VMware vCenter User Manual:
 Go to http://www.emulex.com/products/software-solutions/onecommand-manager-for-vmware-
vcenter/downloads/
 Select VMware
 Select your vSphere version
 In the Documentation column, click on User Manual

At the end of the installation, you should get the following information using the Emulex OneCommand tab:



82
Installing Broadcom FlexFabric dependent hardware iSCSI adapters

Configuring networking for iSCSI

It is necessary to create a vSwitch with the physical NIC associated with the hardware iSCSI adapter found in the Virtual
Connect server profile:
 Log in to the vSphere web Client, and select your host
 Select the Manage menu and click on the Networking tab
 Select the Virtual Switches menu and click on Add host networking

 Select the VMkernel Network Adapter, click Next
 Select New standard switch, click Next
 Click on + to select an adapter
 Select the correct adapter that will be used by the first vmhba

Note: To make sure you select the correct adapter (vmnic) that corresponds to the first hardware iSCSI adapter, open
the server’s VC profile and check the MAC address of the first iSCSI HBA connection:











83
 Click OK

Note: At this step, it is mandatory to select only ONE physical network adapter in order to be eligible for binding to an
iSCSI HBA.

 Click Next




84
 Then enter the applicable settings then click Next

 Enter a static IP address (DHCP is not recommended) and click Next

 Click Finish



85
 Repeat the same configuration steps to create an additional Virtual Switch with the network adapter that will be used
by the second vmhba



Configuring network port binding

Let’s now bind our Accelerated iSCSI adapters to our VMkernel network adapters:
 Select the Storage tab
 Select the Storage Adapters menu
 Select the first Broadcom vmhba and click on Network Port Binding



86
 Click on + to select a VMkernel network adapter

 Select the first VMknernel vSwitch we have just created then click OK

 It is recommended to rescan the storage adapter by clicking on the refresh button

 Repeat the same configuration steps with the second Broadcom vmhba and the second VMknernel vSwitch





87
Configuring the discovery information

Let’s now set up the target discovery address so that iSCSI adapters can determine which storage resource on the
network is available for access:
 From the first Broadcom vmhba, select the Targets tab

 Click on Add… with the Dynamic Discovery option selected

 Enter the IP address of the iSCSI Storage System then click OK


Note: iSCSI SAN vendors usually recommend using a Virtual IP address; please refer to your iSCSI Storage System
documentation.
 Then click on the Rescan Storage Adapter button



88
 You can then click on Devices to verify that the target has been discovered

Newly discovered iSCSI targets should appear in the list if a LUN has been correctly presented to the server.

 Repeat the same configuration steps for the second Broadcom vmhba
 At the end the iSCSI target should be discovered on both ports





89
Checking multipath information

To check the multipathing configuration:
 From the Storage Devices view, select the iSCSI Disk and click on Paths

 Make sure the number of paths displayed is more than one with all paths active:




90
Enabling Flow Control and Jumbo Frames

By default, Flow Control is enabled on every network interfaces so there is no configuration required:

With Broadcom FlexFabric adapters Jumbo Frames must be enabled on vSwitches and VMkernel network adapters.

Enabling Jumbo Frames on vSwitches

To enable Jumbo frames on Accelerated iSCSI virtual switches:
 Click on the Networking tab
 Select the Virtual Switches menu
 Select the vSwitch used by the first hardware iSCSI adapter and click on Edit

 Set the MTU parameter (e.g. 9000) :

 Then click OK


91

Note: When Jumbo Frames for iSCSI traffic is enabled, the network must support Jumbo Frames end-to-end for
Jumbo Frames to be effective.

 Do the same on the second vSwitch used by the second hardware iSCSI adapter


Enabling Jumbo Frames on VMkernel network adapters

To set the VMkernel MTU to match the value configured on the vSwitches:
 Select the VMkernel adapters menu
 Select the VMkernel adapter linked to the vSwitch used by the first hardware iSCSI adapter and click on Edit settings

 Select NIC settings and set the MTU parameter (e.g. 9000) :

 Then click OK
 Then do the same on the second VMkernel adapter linked to the vSwitch used by the second hardware iSCSI adapter





92
Installing Emulex FlexFabric independent hardware iSCSI adapters

Configuring the IP address of the Emulex iSCSI ports

The setup and configuration process for the independent hardware iSCSI adapters involves these steps:
 Log in to the vSphere web Client, and select your host
 Select the Manage menu and click on the Storage Adapters tab
 Select the first Emulex OneConnect… vmhba and click Edit… in the IP address and DNS Configuration section:

 Enter a static IP address (DHCP is not supported) and click OK

 Repeat the same steps for second vmhba


93
Configuring discovery information

Let’s now set up the target discovery address so that iSCSI adapters can determine which storage resource on the
network is available for access:
 From the first Emulex vmhba, select the Targets tab

 Click on Add… with the Dynamic Discovery option selected

 Enter the IP address of the iSCSI Storage System then click OK


Note: iSCSI SAN vendors usually recommend using a Virtual IP address; please refer to your iSCSI Storage System
documentation.



94
 Then click on the Rescan Storage Adapter button

 You can then click on Devices to verify that the target has been discovered

Newly discovered iSCSI targets should appear in the list if a LUN has been correctly presented to the server.

 Repeat the same configuration steps for the second Emulex vmhba
 At the end the iSCSI target should be discovered on both ports





95
Checking multipath information

To check the multipathing configuration:
 From the Storage Devices view, select the iSCSI Disk and click on Paths

 Make sure the number of paths displayed is more than one with all paths active:



Enabling Jumbo Frames and Flow Control on the vmhba adapters

VMware does not support Jumbo frames on Independent Hardware iSCSI Adapter however with Emulex FlexFabric
adapters, the MTU size is auto-negotiated during the TCP connection with the iSCSI target, and therefore there is no
configuration required.
By default, Flow Control is enabled on every network interfaces when Accelerated iSCSI is enabled.
See Enabling Flow Control on Host Accelerated iSCSI adapters and Enabling Jumbo Frames on iSCSI Storage Systems


96
Using the VMware iSCSI Software Initiator in conjunction with Accelerated iSCSI support

The use of an iSCSI Software initiator in conjunction with the iSCSI hardware initiator is possible under VMware. This
section describes the configuration steps required under HP Virtual Connect and VMware vSphere.

Virtual Connect Profile Configuration

From the ESX server’s VC Profile, make sure that at least one server NIC is connected to an iSCSI network or to a network
where an iSCSI device is available:

Note: It is recommended to use a dedicated network adapter for iSCSI with a suitable bandwidth.

vSphere iSCSI Software configuration with multiple NICs

With the software-based iSCSI implementation, we use standard Ethernet adapters (versus iSCSI HBA connections) to
connect the host to an iSCSI target.
 Log in to the vSphere web Client, and select your host
 Select the Manage menu and click on the Networking tab

Software iSCSI
Accelerated iSCSI


97
 Select the Virtual Switches menu and click on Add host networking

 Select the VMkernel Network Adapter, click Next
 Select New standard switch, click Next
 Click on + to select adapters
 Select the two vmnic that will be connected to the iSCSI networks (e.g. iSCSI-1 and iSCSI-2) and click OK

 Click Next



98
 Enter a network label (e.g. iSCSI-Network-1) then click Next

 Specify the IP Settings (Static or DHCP) then click Next
 Click Finish
 Click again on Add host networking

 Select again the VMkernel Network Adapter option, click Next



99
 Now make sure you select the vSwitch that has just been created above and click Next

 Enter a network label (e.g. iSCSI-Network-2) then click Next

 Specify the IP Settings (Static or DHCP) then click Next
 Click Finish




100
Mapping each iSCSI port to just one active NIC

 Select the vSwitch that has just been created and on the diagram, click on iSCSI-Network-1

 Then click on the Edit icon

 Click on Teaming and Failover and select Override




101
 Using the down arrow, move the second vmnic to the Unused adapters section

 Click OK
 Repeat the same steps for iSCSI-Network-2 but this time using the first vmnic as the unused adapter:




102
 Review the configuration then click OK

Note: The iSCSI-Network-1 VMkernel interface network is linked to the first vmnic and iSCSI-Network-2 is linked to
second vmnic.


Enabling Jumbo Frames on vSwitch

To enable Jumbo frames on the vSwitch used by iSCSI-Network-1 and iSCSI-Network-2:
 Select the Virtual Switches menu
 Select the vSwitch linked to the iSCSI-Network-1 and iSCSI-Network-2 and click on Edit settings



103
 Set the MTU parameter (e.g. 9000) :

 Then click OK

Note: When Jumbo Frames for iSCSI traffic is enabled, the network must support Jumbo Frames end-to-end for
Jumbo Frames to be effective.

For more information: See iSCSI and Jumbo Frames configuration on ESX/ESXi (KB: 1007654),
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=100765
4


Enabling Jumbo Frames on VMkernel network adapters

To set the VMkernel MTU to match the value configured on the vSwitch:
 Select the VMkernel adapters menu
 Select the VMkernel adapter linked to the iSCSI-Network-1 and click on Edit settings

 Select NIC settings and set the MTU parameter (e.g. 9000) :



104
 Then click OK
 Then do the same on the second VMkernel adapter linked to the iSCSI-Network-2

Configuring iSCSI Software target discovery address

 Click on the Storage tab
 Select the Storage Adapters menu
 Click on + to add a new adapter

 Select Software iSCSI adapter

 Click OK at the pop-up message

 The new software iSCSI Adapter shows up in the Storage adapters list:



105
IMPORTANT: The host assigns a default iSCSI name to the initiator that must be changed in order to follow the iSCSI
RFC-3720, where it is stated to use the same IQN for all connections for a given host and NOT associate different ones to
different adapters.
To change the iSCSI name of the iSCSI Software adapter in order to use the same IQN for all connections:
 Select the iSCSI Software vmhba adapter then click on the Properties tab
 Click on Edit

 Then enter the same iSCSI name than the other storage adapters and click OK

 At the end, all connections must use the same IQN :





106
Binding iSCSI Ports to the iSCSI Adapters

Let’s now bind our iSCSI Software adapter to our VMkernel network adapters:
 Select the iSCSI Software Adapter vmhba and click on Network Port Binding

 Click on + to select the first VMkernel network adapter that was created

 Select iSCSI-Network-1 then click OK



107
 Click again on + to select the second VMkernel network adapter that was created

Note: One or more VMkernel adapters can be bound to a software iSCSI adapter.

 Select iSCSI-Network-2 then click OK

 Click on the Rescan storage adapter icon to refresh the information



108
Configuring iSCSI Software target discovery address

Let’s now set up the target discovery address so that the iSCSI Software adapter can determine which storage resource
on the network is available for access:
 Select the Targets tab

 Click on Add… with the Dynamic Discovery option selected

 Enter the IP address of the iSCSI Storage System then click OK




109
 Then click on the Rescan Storage Adapter button

 Then click on Devices to verify that the target has been discovered:

 Then select the Paths tab, two active paths must be discovered:



110
 Select Storage Devices menu

Two iSCSI storage targets are now available, one is discovered through Accelerated iSCSI and the other through
Software iSCSI.





111
Boot from iSCSI

The iSCSI Boot feature allows a server to boot from a remote disk (known as the iSCSI target) on the network without
having to directly attach a boot disk.
The main benefits are:
 Centralized boot process
 Cheaper servers (diskless)
 Less server power consumption - Servers can be denser and run cooler without internal storage.
 Boot From SAN-like benefits at attractive costs
 Easier server replacement – You can replace servers in minutes, the new server point to the old boot location.
 Easier backup processes – The system boot images in the SAN can be backed up as part of the overall SAN backup
procedures.

Note: Under Virtual Connect management, Accelerated iSCSI takes automatically place when iSCSI boot is enabled.

Supported Operating Systems for iSCSI Boot
At the time of writing, not all Operating Systems are supported by Broadcom FlexFabric adapters for iSCSI Boot, for more
information about the Supported Operating Systems for iSCSI Boot on Broadcom FlexFabric adapters, see
http://www.broadcom.com/support/ethernet_nic/faq_drivers.php# and look for “What OS are supported with iSCSI
Boot?”

Boot from iSCSI : Creating a Virtual Connect profile

The following steps provide an overview of the procedure to create a Boot from iSCSI Virtual Connect profile:
 Open Virtual Connect manager. From the Menu Define, select Server Profile in order to create a new VC profile.

Note: VCM assigns FCoE connections by default!



112
Note: A server with one FlexFabric adapter can be configured with a unique personality, either all Ethernet, or
Ethernet/iSCSI, or Ethernet/FCoE. Therefore it’s not possible in this case to enable at the same time both FCoE and iSCSI
connections. So the two existing FCoE connection must be deleted. On the other hand, a server with multiple FlexFabric
Adapters can be configured with both iSCSI and FCoE connections mapped to different adapters.
 If you have a unique FlexFabric Adapter, delete the two FCOE connections by clicking on Delete, otherwise jump to the
next bullet:

 In the iSCSI HBA Connections section, click Add

 In the Network Name column, click on Select a network…

 Select your iSCSI dedicated VC network and click OK


Note: The multiple network feature (i.e. when using 802.1Q VLAN tagging) is not supported for iSCSI connections




113
 In the Port Speed column, you can adjust the speed settings (Auto – Preferred – Custom)

 In the Boot Setting column, select Primary


Note: Disabled: Only Accelerated iSCSI is available. Boot is unavailable.
Primary: Enables you to set up a fault-tolerant boot path and displays the screen for Flex-10 iSCSI connections.
Accelerated iSCSI is enabled. USE-BIOS: Indicates if boot will be enabled or disabled using the server iSCSI BIOS utility.

 A new window menu pops-up automatically when Primary is selected:




114
 2 choices are offered at this point:
 Enter all iSCSI boot parameters manually for the primary connection.
Visit the Appendix 1 for help.
 Use the iSCSI Boot Assistant (if previously configured, see below for more details).

Note: Boot Assistant supports only HP StoreVirtual Storage Solution today; it’s the recommended method as it
simplifies the procedure and avoids typing errors.

Boot Assistant prerequisites:
 The IP address of the HP StoreVirtual Storage CMC interface must be accessible from the
same management network where Virtual Connect and Onboard Administrator reside.
 Make sure to update your HP StoreVirtual 4000 Storage solution with the latest software
version (HP StoreVirtual Storage is the new name for HP LeftHand Storage and HP P4000
SAN solutions).
 Servers and volumes must be created on the StoreVirtual storage.
 StoreVirtual volumes must be assigned to servers.
 The StoreVirtual user and password credentials must be configured in VCM:
o Click on the Storage Mgmt Credentials link on the left hand side menu:

o Click Add and enter the required information:






115
Steps with Boot Assistant:
 Click Use Boot Assistant
 Then select the iSCSI target from ‘Management Targets’ drop down list, then click Retrieve

 Then select the Boot Volume



 After all entries have been correctly entered either manually or by the Boot Assistant, save the iSCSI Boot parameters,
by clicking Apply


Note: Boot Assistant will populate information for ‘iSCSI Boot Configuration’ and ‘Authentication’. The user needs to fill
in the ‘Initiator Network Configuration’, either by checking ‘Use DHCP to retrieve network configuration’ or filling in the IP
address and Netmask fields.




116
 Now we can define the second iSCSI connection, in the iSCSI HBA Connections section, click Add

Note: Allowing more than one iSCSI application server to connect to a volume concurrently without cluster-aware
applications or without an iSCSI initiator with Multipath I/O software could result in data corruption.

 In the Network Name column, select your second iSCSI dedicated VC network,

 In the Port Speed column, you can adjust the speed settings (Auto – Preferred – Custom)

 In the Boot Setting column, select Secondary

 A new window menu pops-up automatically when Secondary is selected, enter all iSCSI boot parameters for the
secondary connection, the settings are the same as the primary configuration (you can consult the Appendix 1 for
help) or you can use the iSCSI Boot Assistant (if previously configured).


 Save by clicking Apply.




117
 Additional VC Ethernet Network connections must be added to the VC profile in order to give to the server some other
network access like Service Console, VM Guest VLAN, VMotion, etc. This will obviously depend on your server
application.

 When done, you can assign your profile to a server bay


 Click Apply to save the profile




118
 The server can now be powered on (using either the OA, the iLO, or the Power button)

 During POST, the following information is displayed by the FlexFabric adapter:

 with Emulex adapter:

 with Broadcom adapter:



 Make sure the iSCSI disk target is shown during the iSCSI scan.



119
 Validation of the iSCSI configuration with Emulex adapters:





 Validation of the iSCSI configuration with Broadcom adapters:






 If everything is correct, you can start with the OS deployment, otherwise consult the Troubleshooting section.

iSCSI Disk Volume
Make sure you see the drive
information

Initiator name
iSCSI port 1
Make sure you get an IP
address if DHCP is
enabled
iSCSI port 2
Make sure you get an IP
address if DHCP is
enabled

iSCSI Target name

iSCSI port 1
Make sure you get an IP
address if DHCP is
enabled
Initiator name
Make sure each port logs in the iSCSI
target successfully (2
nd
port is not
shown in this screenshot)
iSCSI Target information



120
Boot from iSCSI: Installing Microsoft Windows Server 2012

A Windows Server iSCSI boot installation does not require any particular instructions except that the iSCSi drivers must
be provided right at the beginning of the Windows installation in order to discover the iSCSI drive presented to the
server.
 Launch the Windows installation

 At the “Where do you want to install Windows?” step, the Windows installation does not detect any iSCSI drive






121
 So it is necessary to provide at this stage, the iSCSI drivers of the Emulex card (NC55x 10Gb FlexFabric Adapter) in
order to detect the iSCSI LUN target.

To obtain these drivers, click below on the web link corresponding to the Emulex card you have in the server:

HP NC551i Dual Port FlexFabric
10Gb Converged Network Adapter
http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownl
oad.jsp?lang=en&cc=us&prodNameId=5033634&taskId=135&prodT
ypeId=329290&prodSeriesId=5033632&lang=en&cc=us
HP NC553i 10Gb 2-port FlexFabric
Server Adapter
http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownl
oad.jsp?lang=en&cc=us&prodNameId=4324854&taskId=135&prodT
ypeId=3709945&prodSeriesId=4296125&lang=en&cc=us
HP FlexFabric 10Gb 2-port 554FLB
Adapter
http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownl
oad.jsp?lang=en&cc=us&prodNameId=5215362&taskId=135&prodT
ypeId=3709945&prodSeriesId=5215333&lang=en&cc=us
HP FlexFabric 10Gb 2-port 554M
Adapter
http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownl
oad.jsp?lang=en&cc=us&prodNameId=5215390&taskId=135&prodT
ypeId=3709945&prodSeriesId=5215389&lang=en&cc=us

Then select Microsoft Windows Server 2012 and download the HP Emulex 10GbE iSCSI Driver:


Note: With Broadcom FlexFabric adapters, this procedure to provide the iSCSI drivers (via USB or CD/DVD) during the
installation as shown below is not supported. It is necessary to inject Broadcom drivers into the Windows installation
image using Windows ADK, for more information, see Injecting (Slipstreaming) Broadcom Drivers into Windows Image
Files in the Broadcom NetXtreme II 10G User Manual,
http://www.broadcom.com/support/ethernet_nic/netxtremeii10.php (look in the User Manual section).

 Using the iLO Virtual Drives menu, mount a virtual folder and select the iSCSI folder located where your HP Emulex
drivers have been unpacked.




122
 Then select Load driver

 And browse the just mounted iLO folder and click OK





123
 Windows should detect the be2iscsi.inf file contained in the folder

 Click Next



124
 The installation should finally detect the iSCSI Boot volume that has been presented to the server

Select the iSCSI Volume and click Next then you can complete the Windows installation.

Note: Sometimes a same drive is detected several times by the Windows installation due to multipathing; this is
because the server profile under VCM is configured with two iSCSI ports. In such case, the Windows installation
automatically turns offline the duplicated drives and leaves only one drive that can be selected.








125
Boot from iSCSI: Installing Microsoft Windows Server 2008

An iSCSI boot Windows Server installation does not require any particular instructions except that the iSCSi drivers must
be provided right at the beginning of the Windows installation in order to discover the iSCSI drive presented to the
server.
 Launch the Windows installation


 At the “Where do you want to install Windows?” stage, the Windows installation does not detect any iSCSI drive


 So it is necessary to provide at this stage, the iSCSI drivers of the Emulex card (NC55x 10Gb FlexFabric Adapter) in
order to detect the iSCSI LUN target.


126

To obtain these drivers, click below on the web link corresponding to the Emulex card you have in the server:

HP NC551i Dual Port FlexFabric
10Gb Converged Network Adapter
http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownl
oad.jsp?lang=en&cc=us&prodNameId=5033634&taskId=135&prodT
ypeId=329290&prodSeriesId=5033632&lang=en&cc=us
HP NC553i 10Gb 2-port FlexFabric
Server Adapter
http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownl
oad.jsp?lang=en&cc=us&prodNameId=4324854&taskId=135&prodT
ypeId=3709945&prodSeriesId=4296125&lang=en&cc=us
HP FlexFabric 10Gb 2-port 554FLB
Adapter
http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownl
oad.jsp?lang=en&cc=us&prodNameId=5215362&taskId=135&prodT
ypeId=3709945&prodSeriesId=5215333&lang=en&cc=us
HP FlexFabric 10Gb 2-port 554M
Adapter
http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownl
oad.jsp?lang=en&cc=us&prodNameId=5215390&taskId=135&prodT
ypeId=3709945&prodSeriesId=5215389&lang=en&cc=us

Then select Microsoft Windows Server 2008 and download the HP Emulex 10GbE iSCSI Driver:


Note: With Broadcom FlexFabric adapters, this procedure to provide the iSCSI drivers (via USB or CD/DVD) during the
installation as shown below is not supported. It is necessary to inject Broadcom drivers into the Windows installation
image using Windows ADK, for more information, see Injecting (Slipstreaming) Broadcom Drivers into Windows Image
Files in the Broadcom NetXtreme II 10G User Manual,
http://www.broadcom.com/support/ethernet_nic/netxtremeii10.php (look in the User Manual section).


 Using the iLO Virtual Drives menu, mount a virtual folder and select the iSCSI folder located where your HP Emulex
drivers have been unpacked.



127
 Then select Load driver

 And browse the just mounted iLO folder and click OK





128
 Windows should detect the be2iscsi.inf file contained in the folder

 Click Next






129
 The installation should finally detect the iSCSI Boot volume that has been presented to the server

 Select the iSCSI Volume and click Next then you can complete the Windows installation.



Note: Sometimes a same drive is detected several times by the Windows installation due to multipathing; this is
because the server profile under VCM is configured with two iSCSI ports. In such case, the Windows installation
automatically turns offline the duplicated drives and leaves only one drive that can be selected.




130
Boot from iSCSI: Installing VMware vSphere 5.x

Prerequisites
 Ensure that the LUN is presented to the ESX system as LUN 0. The host can also boot from LUN 255.
 Ensure that no other system has access to the configured LUN.

Manual ESXi installation Method

The installation of an iSCSI Boot server with ESXi 5.x is quite straight forward and does not required any specific steps:
 Download the installation ISO from the HP website, http://www.hp.com/go/esxidownload
 Burn the installation ISO to a CD, or move the ISO image to a location accessible using the virtual media capabilities of
iLO.
 Boot the server and launch the HP VMware ESXi installation




131
 Click Enter

 Click F11 to accept the agreement




132
 The iSCSI volume to install ESXi should be detected without difficulty by the installer:

Note: At the time of writing, VMware vSphere is not supported by Broadcom FlexFabric adapters for iSCSI Boot, for
more information about the Supported Operating Systems for iSCSI Boot on Broadcom FlexFabric adapters, see
http://www.broadcom.com/support/ethernet_nic/faq_drivers.php# and look for “What OS are supported with
iSCSI Boot?”

 Once the disk is selected, you can carry on with the next steps



133
 At the end, you can reboot the server to start ESXi 5.x

 Once rebooted, it is necessary to install the latest Emulex drivers and firmware, see
http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?objectID=c03005737





134
Boot from iSCSI: Installing Red Hat Enterprise Linux

With the HP FlexFabric 10Gb 2-port 55x Adapters, refer to the Emulex Boot Manual (see Setting Up Boot from SAN for
iSCSI): http://www.emulex.com/files/downloads/hardware/oc_boot_manual.pdf
With the HP FlexFabric 10Gb 2-port 53x Adapters, refer to the NetXtreme II 10G User Manual (see Linux iSCSI Boot Setup):
http://www.broadcom.com/support/ethernet_nic/netxtremeii10.php (go to the User Manual section).


135
Boot from iSCSI: Installing SUSE Linux Enterprise Server

With the HP FlexFabric 10Gb 2-port 55x Adapters, refer to the Emulex Boot Manual (see Setting Up Boot from SAN for
iSCSI): http://www.emulex.com/files/downloads/hardware/oc_boot_manual.pdf
With the HP FlexFabric 10Gb 2-port 53x Adapters, refer to the NetXtreme II 10G User Manual (see Linux iSCSI Boot Setup):
http://www.broadcom.com/support/ethernet_nic/netxtremeii10.php (go to the User Manual section).


136
Troubleshooting

FlexFabric adapter’s iSCSI BIOS Utility is a good place to start troubleshooting iSCSI issues.
Two different iSCSI Bios utilities are available today with FlexFabric adapters:
 Emulex iSCSISelect Utility
 Broadcom Comprehensive Configuration Management (CCM)

Emulex iSCSI Initiator BIOS Utility

With Emulex FlexFabric adapters, the iSCSI Initiator BIOS Utility is called Emulex iSCSISelect Utility.
It is an INT 13H option ROM resident utility that you can use to configure, manage and troubleshoot your iSCSI
configuration.
Note: In order to get into the iSCSI utility, iSCSI must be enabled in the Virtual Connect profile!
For more detailed information, see HP NC-Series ServerEngines iSCSISelect User Guide:
http://h10032.www1.hp.com/ctg/Manual/c02255542.pdf

With Emulex iSCSISelect you can:
 Configure an iSCSI initiator on the network
 Ping targets to determine connectivity with the iSCSI initiator
 Discover and display iSCSI targets and corresponding LUNs
 Initiate boot target discovery through Dynamic Host Configuration Protocol (DHCP)
 Manually configure bootable iSCSI targets
 View initiator properties
 View connected target properties

Note: HP Virtual Connect profile takes precedence over the settings in this menu:
If iSCSI SAN boot settings are made outside of Virtual Connect (using iSCSISelect or other configuration tools), Virtual
Connect will restore the settings defined by the server profile after the server blade completes the next boot cycle. Only
the USE-BIOS option in a VC Profile preserves the boot settings options set in the iSCSI Utility or through other
configuration utilities.




137
To run the Emulex iSCSISelect Utility, press <Ctrl>+<S> when prompted:

The iSCSI Initiator Configuration menu will appear after the BIOS initializes.

The iSCSI Initiator name we get here is related to the control of Virtual Connect.
 A server with a VC profile without iSCSI Boot parameters shows a factory default initiator name like “iqn.1990-
07.com.Emulex.xx-xx-xx-xx-xx“ with in red the default Factory MAC address of the iSCSI adapter.
 A server with a VC profile with iSCSI Boot parameters shows the initiator name defined in the VC Profile.



138
Configuration checking
If you are facing some connection problems with your iSCSI target, you can use the iSCSISelect Ping utility to
troubleshoot your iSCSI configuration.
If you don’t see any iSCSI Boot drive detected during POST (‘BIOS not installed’ is displayed during POST) you might need
to check the iSCSI configuration written by VC.
 From the iSCSI Initiator Configuration menu, tab to Controller Configuration menu, and press Enter.
 Select the first Controller and press Enter.
 Move to the iSCSI Target Configuration and press Enter.

 Verify you have one iSCSI target, this target must correspond to your Virtual Connect profile configuration
 Select the target and press Enter.
 You can verify the target name, the IP Address and the authentication method.



139
A screen without target information like seen below, means that there is an issue somewhere:

Multiple reasons can lead to this issue:
 Out-dated Emulex firmware.
 Authentication information problem.
 Virtual Connect profile not assigned (make sure also there is no issue reported by the VC Domain).
 Network connectivity issue between the server and the iSCSI target.
 iSCSI target problem: misconfiguration, wrong LUN masking, etc.

For further debugging, you can manually enter the target configuration to run a PING test:
 Select Add New iSCSI target
 Enter all the target information




140
Then select Ping and press Enter.

You must get the following successful result:

 If ping is successful, then it’s very likely it’s the incorrect authentication information.
 If you still cannot connect or ping your iSCSI target, please refer to the following paragraphs.





141
Broadcom iSCSI Initiator BIOS Utility

With Broadcom FlexFabric adapters, the iSCSI Initiator BIOS Utility is called Broadcom Comprehensive Configuration
Management (CCM).
Broadcom CCM is an INT 13H option ROM resident utility that you can use to configure, manage your iSCSI configuration.
Broadcom CCM provides a menu mode and a scripting mode.
For more detailed information, see Broadcom Comprehensive Configuration Management for HP FlexFabric Adapters User
Guide: http://h10032.www1.hp.com/ctg/Manual/c03923035.pdf

Note: In order to get into the iSCSI utility, iSCSI must be enabled in the Virtual Connect profile!
Note: HP Virtual Connect profile takes precedence over the settings in this menu:
If iSCSI SAN boot settings are made outside of Virtual Connect (using CCM or other configuration tools), Virtual Connect
will restore the settings defined by the server profile after the server blade completes the next boot cycle. Only the USE-
BIOS option in a VC Profile preserves the boot settings options set in the iSCSI Utility or through other configuration
utilities.

To run the Broadcom CCM, press <Ctrl>+<S> or the <Ctrl>+<B> keys (depending on the configuration) when prompted:




142
The Main Menu describes the configuration options available for the device selected in the Device List menu.

The iSCSI Boot Main Menu describes the iSCSI Boot configuration options.




143
The General Parameters menu describes the iSCSI boot general parameters available for configuration.

The Initiator Parameters menu describes the iSCSI boot target parameters.






144
Emulex OneCommand Manager (OCM)

The Emulex OneCommand Manager Application is a comprehensive management utility for Emulex and converged
network adapters (CNAs) and host bus adapters (HBAs) that provides a powerful, centralized adapter management suite,
including discovery, reporting. It’s an excellent tool for troubleshooting. OneCommand Manager contains a graphical
User Interface (GUI) and a Command Line Interface (CLI)
OneCommand Manager is available under Windows, Linux, VMware vSphere, etc.

For more information about OneCommand Manager, see Emulex OneCommand Manager User Manual and Release Notes:
http://www.emulex.com/products/software-solutions/onecommand-manager/downloads/ (select any Windows OS)

Some of the OneCommand Manager interesting features available for iSCSI:
 Run diagnostic tests on adapters
 Reset / disable adapters
 Manage an adapter's CEE settings
 Discover iSCSI targets
 Login to iSCSI Targets from CNAs
 View iSCSI Target session information
 Logout from iSCSI targets

Not all OneCommand Manager Application features are supported across all operating systems. Consult the Emulex
OneCommand documentation.


3 NIC
1 iSCSI
0 FCoE
Target LUN 0 from Port 2
Target LUN 0 from Port 1
MAC of the NICs
iSCSI Target
iSCSI Initiator


145
iSCSI LUN information

If you select the target, you can click on Sessions





146
And get lots of diagnostic of the iSCSI target session


Diagnostics



147
Broadcom Advanced Control Suite (BACS)

Broadcom Advanced Control Suite is the Broadcom utility to manage the FlexFabric 53x and 63x Converged Network
Adapters. Among other things, it provides comprehensive control of the iSCSI network including discovery, reporting and
settings.
For more information about Broadcom Advanced Control Suite and Broadcom Adapters, refer to:
 The NetXtreme II 10G User Manual:
http://www.broadcom.com/support/ethernet_nic/netxtremeii10.php (go to the User Manual section)
 Broadcom Advanced Control Suite web page :
http://www.broadcom.com/support/ethernet_nic/management_applications.php
 HP FlexFabric and StoreFabric Adapters User Guide: http://h10032.www1.hp.com/ctg/Manual/c03916480.pdf

To download the Broadcom Advanced Control Suite (BACS), go to the Broadcom web site:
http://www.broadcom.com/support/ethernet_nic/netxtremeii10.php#mgmt_apps

BACS includes a graphical user interface and a command line interface (BACSCLI). BACS GUI and BACS CLI can operate on
the following operating system families:
 Windows and Windows Server
 Linux Server

Graphical User Interface




Target LUN 0 from Port 2
Target LUN 0 from Port 1
iSCSI Port 1
iSCSI Target
iSCSI Initiator
iSCSI Port 2


148
iSCSI LUN Information

iSCSI Target Information




149
Port Information




150
Problems found with OneCommand Manager

Problem:
You have configured a VC server profile with two iSCSI ports but OneCommand Manager or the iSCSI Initiator show
only one iSCSI port connected to the iSCSI LUN




Solution:
To properly configure iSCSI load balancing sometimes like with HP StoreVirtual 4000 Storage, you must use for the
Primary Target Address in the iSCSI boot parameters the Virtual IP Address (VIP) of the cluster and not the IP
address of the HP StoreVirtual Storage node.


Also verify with your Storage Administrator that all appropriate clusters have VIPs configured.
Make sure you use the VIP of
your storage cluster when using
two iSCSI ports boot connections


151
Problems found during iSCSI Boot
Problem:
You are using a DHCP server to get the iSCSI Boot parameters but during POST (server Power-On Self-Test) the iSCSI
disk is not be displayed when the iSCSI Initiator is executed although the menu shows correct initiator IP addresses:

Solution:
 You are not able to get the DHCP option 43 to work with the FlexFabric 10Gb Converged Network Adapters, for
more information see Appendix 2.

 Make sure the storage presentation is correctly configured on the storage array.


Problem:
You get a warning message with IP address: 0.0.0.0

Solution:
There are different reasons for this issue:
 Your second iSCSI boot connection is not configured in the VC profile, in which case it’s not a problem.
 You have a network connection issue; check the VC profile and the vnet corresponding to the iSCSI connection.
Verify the status of the vnet and make sure it is properly configured with all ports in green.


152

Problem:
Emulex is showing “BIOS Not Installed” with no drive information:

Broadcom is not showing “Logging in the iSCSI Target… Succeeded” with no driver information:


Solution:
 Make sure the Storage presentation (also known as LUN masking) is correct.
 Make sure the FlexFabric adapter is using the latest Firmware.

Broadcom showing a successful connection:


153
 This may also be network, or authentication information problem:

With iSCSISelect for Emulex adapters:
 Check in Controller Properties page to make sure Boot Support is enabled.

 Check the network in iSCSISelect in Network Configuration page, make sure Link Status field is ‘Link
Up’ and DHCP field match what is configured in VC.

 Click on Ping




154
 And enter the ‘target IP’ to make sure the network connection is OK.

 If ping failed and DHCP is enabled, check the DHCP server setup.
 If ping failed and DHCP is disabled:
 Select Configure Static IP Address

 Then make sure the initiator IP, network mask and gateway are set up correctly:

 If Ping is successful, make sure authentication configuration in VC matches what is configured on
storage side.



155
With Comprehensive Configuration Management for Broadcom Adapters:
 Check the iSCSI Boot Configuration

 Check in the General Parameters page to make sure HBA Boot Mode is enabled.

 Check in 1st Target Parameters to make sure Connect is enabled.





156

Problem:
You cannot boot from iSCSI and your boot iSCSI LUN ID is above 8.
Solution:
Change the iSCSI boot LUN ID to a number lower than 9.


PXE booting problems

Problem:
Media test failure when PXE booting a G6 server with a CNA mezzanine and only FlexFabric modules in bay 3 and 4.
Solution:
Make sure all the LOM ports (i.e. 1 and 2) are PXE disabled in the VC profile otherwise you will see the PXE error
media test failure because the default VC settings are Use-Bios:






157
iSCSI boot install problems with Windows Server

Problem:
Drivers loaded successfully, target LUN seen, installation proceeded to target LUN, no complaint of drivers not
found, but blue screen occurred.
Solution:
Monitor the network to the iSCSI storage using one of the multiple network monitoring tools available on the
market. This used to be one of the causes for iSCSI boot install problems with Windows 2008. For more information,
see the Appendix 3: How to monitor an iSCSI Network.


Problem:
‘Load driver’ seem to hang. It should only take a few seconds.
Solution:
Check if the firmware version matches driver version. It may hang if not match.


Problem:
During ‘Expanding files’ phase, installation complained it could not locate some files and terminated.
Solution:
Monitor the network access to the iSCSI storage. Network can be a cause.



158
Problem:
iSCSI boot parameters are set up under VCM, but iSCSISelect/Comprehensive Configuration Management utility do
not show configurations the iSCSI Target Configuration for the specified controller.
Solution:
Make sure the VC Network Configuration for this adapter port is correct. Check link status, make sure it’s ‘Link Up’.
Make sure the IP, Mask, and Gateway are correct (if routing is required from initiator to target).



VCEM issues with Accelerated iSCSI Boot

Problem:
During the creation of an iSCSI boot Virtual Connect profile under VCEM 6.2 and you get an unclear format error
message.
Solution:
VCEM 6.2 only allows all LOWERCASE for the initiator and target information (does not apply to VCM).




iSCSI issues with HP StoreVirtual 4000 Storage

Problem:
You are connected to a HP StoreVirtual 4000 Storage Solution using an Emulex FlexFabric adapter and during heavy
network utilization; the iSCSI initiator driver reports a device failure.
Solution:
 There is something that is significantly slowing down the response time on the network such that the iSCSI
initiator session recovery can’t occur within the default Emulex adapter driver settings. In this case, it might be
useful to increase the iSCSI initiator driver timeout parameter called ETO (or Extended TimeOut) and is
configurable via OneCommand Manager.
 By default ETO is 90 seconds on all Windows operating systems. It can be set between 20 and 3600 seconds.
You can set it to 0 also, but the minimum value assumed by the driver is 20.






159
 To set the value:
 Launch OneCommand Manager.

 Click on the first iSCSI port and select the iSCSI target.


 In the ETO entry, enter the new timeout value and click Apply

 Do the same on the second iSCSI port.





160

Appendix 1- iSCSI Boot Parameters

Mandatory iSCSI Boot Parameters entries
Some entries have to be correctly filled up in order to successfully boot from iSCSI.


Example of a correctly configured iSCSI Boot configuration:



161
iSCSI Initiator (iSCSI Boot Configuration)
(also known as the iSCSI client or iSCSI host)


Name used for the iSCSI initiator on the booting system. The initiator name length can be maximum of 223 characters.


Note: Make sure the Initiator Name you set is unique. This Initiator Name must be given to the Storage Administrator for
the Storage presentation (also known as LUN masking).
Note: Use the same IQN initiator name for all connections for a given host and NOT associate different ones to different
adapters. (Ref. iSCSI RFC-3720).

Each iSCSI host is identified by a unique iSCSI qualified name (IQN). This name is similar to the WorldWide Name (WWN)
associated with Fibre Channel devices and is used as a way to universally identify the device.
iSCSI qualified names take the form iqn.yyyy-mm.naming-authority:unique name, where:
 yyyy-mm is the year and month when the naming authority was established.
 naming-authority is usually reverse syntax of the Internet domain name of the naming authority. For example, the
iscsi.vmware.com naming authority could have the iSCSI qualified name form of iqn.1998-01.com.vmware.iscsi. The
name indicates that the vmware.com domain name was registered in January of 1998, and iscsi is a subdomain,
maintained by vmware.com.
 unique name is any name you want to use, for example, the name of your host.

IQN examples:
iqn.1991-05.com.microsoft:win-g19b6w8hsum
iqn.1990-07.com.Emulex.00:17:A4:77:04:02
iqn.1998-01.com.vmware.iscsi:name1
iqn.1998-01.com.vmware.iscsi:name2




162
iSCSI Target (iSCSI Boot Configuration)

The iSCSI target parameters can be set either statically (by disabling DHCP) or dynamically (by enabling DHCP). For
simplification and human error prevention, it’s may be easier to use a DHCP server for the iSCSI boot parameters.

Configuring manually the target information

 The Target Name

The Target Name is your iSCSI target from which the server boots. The target name length is of maximum 223
characters.
 This target name is provided by the Storage Administrator

 The Boot LUN

The LUN of the Target identifies the volume to be accessed. Valid values for standard LUNs are 0-255 decimal. Valid
values for extended LUNs are 13 to 16 character hexadecimal values.
 This Boot LUN is provided by the Storage Administrator

 The Primary Target Address

The Primary Target Address is the primary IP address used by the iSCSI target.
The TCP port associated with the primary target IP address is by default 3260.

 This Primary Target Address and TCP Port is also provided by the Storage Administrator.

 Depending on your storage solutions, but sometimes if you plan to configured a second iSCSI boot
connection from the second iSCSI HBA, it is required to enter here the Virtual IP Address (VIP) of your storage
cluster and not the IP address of one of the node (that’s the case with HP StoreVirtual 4000 Storage
Solutions)

Note: A virtual IP (VIP) address is a highly available IP address which ensures that if a storage node in a cluster
becomes unavailable, servers can still access a volume through the other storage nodes in the cluster.

The benefits of using a VIP here, is that in case of a storage node failure, the iSCSI traffic does not failover to the
second iSCSI port thus reducing risk, latency, etc.




163
Configuring dynamically the target information

 To use DHCP when configuring the iSCSI boot configuration, select the Use DHCP checkbox.

Note: In a dynamic configuration the initiator name, the target name, the LUN number and the IP address can be
provided by the DHCP server. Selecting this option requires a DHCP server to be set up properly with iSCSI extensions
to provide boot parameters to servers. Refer to Appendix 2.


Initiator Network Configuration

 The VLAN ID

This is the VLAN number that the iSCSI initiator will use for all sent and received packets. Accepted value is from 1 to
4094.

Note: The multiple networks feature being not supported for iSCSI HBA, the VLAN ID field should be left blank for all
scenarios presented in this cookbook (i.e. scenario 1, scenario 2 and scenario 3).
The ‘VLAN id’ option can be required when Virtual Connect is tunneling the iSCSI traffic (i.e. when the iSCSI network
under VCM has the ‘Enable VLAN Tunneling’ checkbox selected). The use of a Tunnel network for the iSCSI traffic is not
presented in this cookbook because the most common scenarios are using the iSCSI vNet in VLAN mapping mode (i.e.
not using the VLAN tunneling mode).


 The IP address/Netmask/Gateway

This is the network configuration of the iSCSI initiator, a fixed IP address or DHCP can be either set.

The Use-DHCP checkbox allows the iSCSI option ROM to retrieve the TCP/IP parameters from a DHCP server.



164
Optional iSCSI Boot Parameters entries
Some fields are optional and can be configured for enhancements.

Secondary iSCSI Target Address

Secondary iSCSI target address is used in the event primary target IP network is down. If primary target fails to boot,
Secondary iSCSI boot is used instead.

The TCP port associated with the secondary target IP address is by default 3260.
 This Secondary Target Address and TCP Port is provided by the Storage Administrator
 A secondary target here is usually not required for a server with a second CNA iSCSI port


Security enhancement using an authentication

For added network security, the Challenge Handshake Authentication Protocol (CHAP) can be enabled to authenticate
initiators and targets. By using a challenge/response security mechanism, CHAP periodically verifies the identity of the
initiator. This authentication method depends on a secret known only to the initiator and the target. Although the
authentication can be One-Way, you can negotiate CHAP in both directions (2-way CHAP) with the help of the same
secret set for Mutual authentication. You must make sure however, that what you have configured on the target side, is
going to match the initiator side. Both One-Way (CHAP Mode) and Mutual authentication (CHAPM) are supported.

To enable One-Way CHAP authentication:
 Select CHAP then enter the Target CHAP Username and Target Secret

Note: The Target/Initiator CHAP Name and Target/Initiator Secret can be any name or sequence of numbers over 12
and less than 16 characters. However, the username and secret on the Target side must match the name on the
Initiator side.




165
Note: CHAP authentication must also be enabled on the storage system side.



To enable Mutual CHAP authentication:
 Select CHAPM then enter the Target CHAP Username, Target Secret, Mutual Username (known as well as Initiator
CHAP Name), and Mutual Secret (known as well as Initiator Secret).

Note: The Target/Initiator CHAP Name and Target/Initiator Secret can be any name or sequence of numbers over 12
and less than 16 characters. However, the name and secret on the Target side must match the name on the Initiator
side.

Note: If users enabled CHAP/CHAPM through DHCP Vendor Option 43, the username and secret will need to be
entered through VC profile configuration.




166
Note: CHAPM authentication must also be enabled on the storage system side.



167
Appendix 2 - Dynamic configuration of the iSCSI Boot
Parameters
The dynamic configuration of the iSCSI target parameters through a DHCP server is one of the options available in the
iSCSI Boot configuration in order to get automatically all required parameters: initiator name, target name, its IP
address, its ports and Boot LUN number.
During this dynamic configuration, DHCP clients (i.e. iSCSI initiators configured with DHCP enabled for the iSCSI Boot
configuration) are identified by the DHCP server by their vendor type. This mechanism is using the DHCP Vendor Option
43 which contains the vendor specific information. If the DHCP option 43 is not defined as specified by the vendors then
the DHCP server has no possibility to identify the client and therefore will not provide the iSCSI target information.
When this occurs the iSCSI disk will not be displayed when the iSCSI Initiator is executed during POST.

So it’s necessary to configure properly the DHCP server with the mandatory fields provided by the FlexFabric adapter
vendors.
In this section, we will describe the configuration steps required with Emulex FlexFabric adapters for both Windows and
Linux DHCP servers. With Broadcom FlexFabric adapters, see the Broadcom NetXtreme II 10G User Manual:
http://www.broadcom.com/support/ethernet_nic/netxtremeii10.php (go to the User Manual section)






168
Windows 2008 DHCP server configuration
The following is the set up instruction for a Windows 2008 DHCP server to enable DHCP option 43 as specified by Emulex
for the NC55x Dual Port FlexFabric 10Gb Converged Network Adapter.
 Start DHCP Manager.
 In the console tree, click the applicable DHCP server branch.
 Right-click on IPV4 and then click Define Vendor Classes to create a new vendor class.

 Click Add.




169
 In the New Class dialog box, type a descriptive identifying name for the new option in the Display name box. You may
also add additional information to the Description box.

 Type in the data to be used by the DHCP Server service under ASCII (32 characters max) like Emulex
To enter the ID, click the right side of the text box under ASCII.


This ID is the value to be filled in the VC iSCSI Boot Configuration ‘DHCP Vendor ID’ field.



170
 Click OK, and then click Close.

 Right-click on IPV4 again and then click Set Predefined Options to set the vendor specific options.

 In Option class, choose the vendor created above.




171
 Click Add

 Then provide:
 An option name, e.g. "Boot target"
 Data type = String or Encapsulated
 Code = 43
 Add any description.




172
 Click OK

 Click OK again.
 In DHCP Manager, double-click the appropriate DHCP scope.
 Right-click Scope Options and then click Configure Options.

 Click the Advanced tab.




173
 Now choose the vendor class we just entered above (e.g. Emulex)

 Select the check box for 43 (created earlier) and enter the correct string.
(e.g. iscsi:”192.168.1.19”:::”iqn.2003-10.com.lefthandnetworks:p4300:13392:esx5-1”:"iqn.1991-
05.com.vmware:esx5-1":::”D”)

For more information about the format of the DHCP Option 43 string, see below.

 Click OK.

Note: It’s not necessary to restart the DHCP service after making the modification.






174
 After the configuration is complete on the DHCP server, you need to enable DHCP and use the same vendor ID within
the VC server profile in VCM:

Note: Be careful, the DHCP Vendor ID is case sensitive.


Note: If the Initiator Name is not set up in the DHCP Vendor option 43, it IS necessary to enter the initiator name at
the same time as the Vendor ID:


Note: If the Initiator Name is set up in the DHCP Vendor option 43, it IS NOT necessary to enter the initiator name. If
users enter the initiator name then DHCP option 43 set up overrides the value entered from VC.


Note: The precedence of initiator name is: DHCP option 43 then VC configuration then default initiator name.



175
 The Initiator Network Configuration is not configured through DHCP Vendor Option 43, user needs to fill in
the IP address and Netmask fields:

 Or check Use DHCP to retrieve network configuration:

 The Authentication method needs to match to what is configured in DHCP vendor option 43 (i.e. in the
AuthenticationType field).


Note: DHCP vendor option 43 does not configure the Username and secret so make sure to provide
Username/Secret here if authentication is enabled in DHCP:









176

option vendor-class-identifier "Emulex";
option space HP;
option HP.root-path code 201 = text;

class "vendor-classes" {
match option vendor-class-identifier;
}

subclass "vendor-classes" "Emulex" {
vendor-option-space HP;
option HP.root-path "iscsi:\"10.10.10.203\":\"3260\"::\"iqn.19860-
3.com.hp:storage.p2000g3.1020108c44\":::::";

}
Linux DHCP server configuration
The following is the set up instruction for a Linux DHCP server to enable DHCP option 43 as specified by Emulex for the
Emulex Dual Port FlexFabric 10Gb Converged Network Adapter.
For a Linus DHCP server, the vendor option 43 is defined in the /etc/dhcpd.conf file.
The example below illustrates how to define the vendor option 43 for the FlexFabric Adapters in dhcpd.conf:
























After the dhcpd.conf file has been modified and saved, restart the DHCP service, enter:
[root@DHCP_server]# service dhcpd restart
The DHCP server configuration is now complete.




This is the vendor
identifier that must
be set as well in the
iSCSI VC profile
The string format must be defined
correctly as shown. For more
information about the format of the
DHCP Option 43 string, see below.



177
Format of DHCP option 43 for the Emulex FlexFabric Adapters

‘iscsi:’<TargetIP>’:’<TargetTCPPort>’:’<LUN>’:’<TargetName>’:’<InitiatorName>’:’<HeaderDigest>’:’<DataDigest>’:’<
AuthenticationType>’
 Strings shown in quotes are part of the syntax and are mandatory.
 Fields enclosed in angular brackets (including the angular brackets) should be replaced with their corresponding
values. Some of these fields are optional and may be skipped.
 When specified, the value of each parameter should be enclosed in double quotes. See examples below.
 All options are case insensitive.

Description of Optional Parameters
 TargetIP
Replace this parameter with a valid IPv4 address in dotted decimal notation. This is a mandatory field.
 TargetTCPPort
Replace this parameter with a decimal number ranging from 1 to 65535 (inclusive). It is an optional field. The default
TCP port 3260 is assumed, if not specified.
 LUN
This parameter is a hexadecimal representation of Logical Unit number of the boot device. It is an optional field. If
not provided, LUN 0 is assumed to be the boot LUN. It is an eight-byte number which should be specified as a
hexadecimal number consisting of 16 digits, with an appropriate number of 0’s padded to the left, if required.
 TargetName
Replace this parameter with a valid iSCSI target ‘iqn’ name of up to 223 characters. This is a mandatory field.
 InitiatorName
Replace this parameter with a valid iSCSI ‘iqn’ name of up to 223 characters. This is an optional field. If not provided,
the initiator name configured through VC will be used, if it is not configured in VC, then the default Initiator name will
be used.
 HeaderDigest
This is an optional field. Replace this parameter with either “E” or “D”.
 “E” denotes header digest is enabled
 “D” denotes that it is disabled.
If skipped, it is assumed that Header Digest is disabled.
 DataDigest
This is an optional field. Replace this parameter with either “E” or “D”.
 “E” denotes data digest is enabled and
 “D” denotes that it is disabled.
If not provided, it is assumed that Data Digest is disabled by default.
 AuthenticationType
This is an optional field. If applicable, replace this parameter with “D”, “E” or “M”.
 “D” denotes authentication is disabled.
 “E” denotes that One-way CHAP is enabled - the username and secret to be used for one way CHAP must be
specified by non-DHCP means.
 “M” denotes that MutualCHAP is enabled - username and secret required for mutual CHAP authentication
must be specified by non-DHCP means.
If not specified, this field defaults to authentication-disabled.





178
Examples

Note: Emulex requires that all attributes are within double quotes (“) and that any optional parameter not defined must
include a colon (:) in the string. If the string is not properly formed then the option ROM ignores the DHCP servers offer.

‘iscsi:’<TargetIP>’:’<TargetTCPPort>’:’<LUN>’:’<TargetName>’:’<InitiatorName>’:’<HeaderDigest>’:’<DataDigest>’:’<
AuthenticationType>’

Example with target and initiator name:

iscsi:”192.168.1.19”:::”iqn.2003-10.com.lefthandnetworks:p4300:13392:esx5-1”:"iqn.1991-05.com.vmware:esx5-
1":::”D”
 Target IP address: 192.168.1.19
 Target TCP port: Not specified, use default from RFC 3720 (3260)
 Target boot LUN: Not specified, Assumed LUN 0.
 Target iqn name: iqn.2003-10.com.lefthandnetworks:p4300:13392:esx5-1
 Initiator name: iqn.1991-05.com.vmware:esx5-1
 Header Digest : Not specified. Assume disabled.
 Data digest: Not specified. Assume disabled.
 Authentication Type: Disabled.

Default Initiator name and Data Digest settings:

iscsi:”192.168.0.2”:”3261”:”000000000000000E”:”iqn.2009-4.com:1234567890”::”E”::”E”
 Target IP address: 192.168.0.2
 Target TCP port: 3261
 Target boot LUN: 0x0E
 Target iqn name: iqn.2009-04.com:1234567890
 Initiator name: Not specified. Use the Initiator name already configured. Use the default name if none was
configured.
 Header Digest : Enabled
 Data digest: Not specified. Assume disabled.
 Authentication Type: 1-way CHAP.

Default TCP Port and Mutual CHAP:


iscsi:”192.168.0.2”::”000000000000000E”:”iqn.2009-4.com:1234567890”::”E”:”D”:”M”
 Target IP address: 192.168.0.2
 Target TCP port: Use default from RFC 3720 (3260)
 Target boot LUN: 0x0E
 Target iqn name: iqn.2009-04.com:1234567890
 Initiator name: Not specified. Use the Initiator name already configured. Use the default name if none was
configured.
 Header Digest : Enabled
 Data digest: Data Digest disabled
 Authentication Type: Mutual CHAP


179
Appendix 3- How to monitor an iSCSI Network?

Some methods for identifying the root cause of performance problems, bottlenecks and various network issues is
something important to cover when playing with iSCSI traffic.
Let’s see briefly how we can identify network problems and recommend actions.

Monitoring Disk Throughput on the iSCSI Storage System
iSCSI Devices usually provide performance information from their management software:


The performance monitoring on the iSCSI system can help you to understand the current load on the SAN to provide
additional data to support decisions on issues such as the following:
 Configuration options (Would network bonding help me?)
 Capacity expansion (Should I add more storage systems?)
 Data placement (Should this volume be on my SATA or SAS cluster?).

The performance data does not directly provide answers, but will let you analyze what is happening and provide support
for these types of decisions.







180
Monitoring Network and Disk Throughput on the iSCSI Host

VMware vSphere

With vSphere, network performance and disk throughput are displayed from Performance tab, Advanced for both
vSphere Host and Virtual Machines:

To check the different load, you can then switch from Network, Disk, Datastore and Storage Adapter

Virtual Machines vSphere host




181
The Disk monitoring counters:
 Disk Read rate
 Disk Write rate
 Disk Usage


For more information about data analysis and recommended actions to correct the problems you identify, refer to the
“Storage Analysis” page 8 and “Network Analysis” page 11 of the following VMware technical Note:
http://www.vmware.com/files/pdf/perf_analysis_methods_tn.pdf





182
Microsoft Windows Resource Monitor

For Windows System, Microsoft provides a Windows Resource Monitor tool that provides monitoring resource usage in
real time. Among other things, it can help you analyze the disk and network throughput.





















Other performance utility are also provided by Microsoft like SQLIO, designed to measure the I/O capacity of a given SQL
configuration, can help to verify that your I/O subsystem is functioning correctly under heavy loads.
SQL Deployments and Tests in an iSCSI SAN
http://technet.microsoft.com/en-us/library/bb649502%28SQL.90%29.aspx
For MS Exchange, Microsoft provides as well specific tools and counters to measure performance, see
http://technet.microsoft.com/en-us/library/dd351197.aspx








183
Analyzing Network information from the Virtual Connect interface

The Virtual Connect Manager interface provides lots of counters, port statistics and information that can help to
troubleshoot a network issue. Both the GUI and CLI provide telemetry statistics.
From the Interconnect Bays link on the left navigation menu, select one of the modules:

Scroll down to the Uplink Port Information section:

This screen provides details on Port Information (uplinks and downlinks) Port Status, Port Statistics, and Remote Device
Information.


184
The Connected to column shows the upstream device information (LLDP must be enabled on the switch)

Click on Detailed Statistics/Information for the uplink port used for iSCSI:

A statistic screen opens with lot of information; let’s see briefly the port statistics section:











No error must be detected here




No error must be detected here

No error must be detected here

Number of frames discarded due to excessive transit delay

Number of frames discarded due to excessive MTU size
(can indicate Jumbo frames configuration issue)

High number may show that VC is running out of resource



Show FCS error (can indicate a collision issue, use smaller
network segment, avoid hubs)


185









You can refer to the Virtual Connect User Guide for more details about the different port statistics counters.




Show good Jumbo frames statistics
Show Jumbo frames statistics with bad FCS (indicates
collisions?)

Lowest number of error must be shown here








Lowest number of error must be shown here

Frames received that exceeds the maximum permitted
frame size





Excessive PAUSE frames is likely due to network congestion



186
At the end of the screen, the Remote Device Information is also good for upstream switch information sometimes useful
if you need to troubleshoot the remote network devices. The remote switch IP address, MAC address, remote port and
switch description are provided:

VCM provides as well statistics for the FlexNIC ports. Go back to the VC Module Bay1 screen and click on Detailed
statistics for the server port you are using:

Note: The statistics on individual FlexNIC is available since VC 3.70.




187
Analyzing Virtual Connect Network performance

To verify the throughput statistics of a VC port, from the VC CLI, enter:
-> show statistics-throughput <EnclosureID>:<BayNumber>:<PortLabel>

This command shows the historical throughput (bandwidth and packets) of a port.

Note: The same throughput statistics are also available in the GUI since VC 3.70 (from Tools / Throughput statistics…)




188
VC provides also real-time traffic SNMP statistics for all ports: server ports and uplink ports. Those performance stats
can be sent from VC to any SNMP tools (e.g. HP IMC, CACTI, NAGIOS, PRTG, etc.)

CACTI Management Interface





189
Wireshark

Currently the best noncommercial tool for debugging network problems is the open source protocol analysis tool
Wireshark. Wireshark contains protocol dissectors for the iSCSI protocol, which can be useful for debugging network and
performance problems.
Wireshark is available under Windows, Linux, HP-UX, etc. For more information see http://www.wireshark.org/

Software iSCSI analysis

For Software iSCSI traffic, i.e. iSCSI traffic is using a network Interface card; you have to select under Wireshark the
correct interface running the iSCSI traffic for the network capture.
If you are unsure, you can open the VC server profile to get the MAC addresses used by iSCSI

Then under Wireshark, select one of the two interfaces used by MPIO and click Start. You can use the Details button to
see the corresponding interface MAC address



190

For a better display of the iSCSI traffic, enter iSCSI in the Filter field:

Here is a sample of a network capture filtering the iSCSI protocol:






191
iSCSI analysis for Accelerated iSCSI adapters

For Accelerated iSCSI connections (i.e. iSCSI traffic is using a Host Bus Adapter) Wireshark cannot be used on the server
as it captures only packet data from a network interface so it is necessary to use the Virtual Connect Port monitoring
feature to monitor the Accelerated iSCSI traffic.

VC profile using Accelerated iSCSI adapters

The Virtual Connect Port monitoring enables network traffic on a set of server ports to be duplicated to an unused uplink
port so that network traffic on those server ports can be monitored by Wireshark and debugged.
To configure Port Monitoring under VC:
 Open VC Manager
 From the left navigation menu, select Ethernet then Port Monitoring.
 Select an unused uplink port for the Network Analyzer Port
 For the ports to monitor, select the correct downlink used by your server from the All Ports drop down menu:




192
 Then from the new list select the two downlinks connected to VC Bay 1 and Bay 2:

 Click OK
 Enable Port Monitoring and click Apply

 You should see a new icon appearing in the interface meaning port monitoring is running:



193
 Now you are ready to connect a laptop running Wireshark to the VC uplink port configured as the network analyzer
port
 Start the capture and filter using iSCSI




194
Acronyms and abbreviations
Term Definition
BIOS Basic Input/Output System
CLI Command Line Interface
CNA Converged Network Adapter
DCB Data Center Bridging (new enhanced lossless Ethernet fabric)
GUI Graphical User Interface
Flex-10 NIC Port* A physical 10Gb port that is capable of being partitioned into 4 Flex NICs
Flex HBA**
Physical function 2 or a FlexFabric CNA can act as either an Ethernet NIC, FCoE
connection or iSCSI NIC with boot and iSCSI offload capabilities.
HBA Host Bus Adapter
I/O Input / Output
IOS Cisco OS (originally Internetwork Operating System)
IP Internet Protocol
iSCSI Internet Small Computer System Interface
LACP Link Aggregation Control Protocol (see IEEE802.3ad)
LOM LAN-on-Motherboard. Embedded network adapter on the system board
LUN Logical Unit Number
MPIO Multipath I/O
MZ1 or MEZZ1; LOM Mezzanine Slot 1; (LOM) LAN Motherboard/Systemboard NIC
OS Operating System
POST Power-On Self-Test
ROM Read-only memory
SAN Storage Area Network
SCSI Small Computer System Interface
VC Virtual Connect
VC-FC Virtual Connect Fibre Channel module
VCM Virtual Connect Manager
VLAN Virtual Local-area network
vNIC Virtual NIC port. A software-based NIC used by Virtualization Managers
vNet Virtual Connect Network used to connect server NICs to the external Network

*This feature was added for Virtual Connect Flex-10
**This feature was added for Virtual Connect FlexFabric












195
Support and Other Resources
Contacting HP
Before you contact HP
Be sure to have the following information available before you call contact HP:
 Technical support registration number (if applicable)
 Product serial number
 Product model name and number
 Product identification number
 Applicable error message
 Add-on boards or hardware
 Third-party hardware or software
 Operating system type and revision level

HP contact information
For help with HP Virtual Connect, see the HP Virtual Connect webpage: http://ww.hp.com/go/virtualconnect
For the name of the nearest HP authorized reseller:
See the Contact HP worldwide (in English) webpage:
http://www.hp.com/country/us/en/wwcontact.html
For HP technical support:
In the United States, for contact options see the Contact HP United States webpage:
http://welcome.hp.com/country/us/en/contact_us.html
To contact HP by phone:
 Call 1-800-HP-INVENT (1-800-474-6836). This service is available 24 hours a day, 7days a week. For continuous
quality improvement, calls may be recorded or monitored.
 If you have purchased a Care Pack (service upgrade), call 1-800-633-3600. For more information about Care Packs,
refer to the HP website:
http://www.hp.com/hps
 In other locations, see the Contact HP worldwide (in English) webpage:
http://welcome.hp.com/country/us/en/wwcontact.html

Subscription service
HP recommends that you register your product at the Subscriber's Choice for Business website:
http://www.hp.com/country/us/en/contact_us.html
After registering, you will receive email notification of product enhancements, new driver versions, firmware updates,
and other product resources.





196
Related documentation
HP Virtual Connect Manager 4.20 Release Notes
https://h20566.www2.hp.com/portal/site/hpsc/template.PAGE/public/kb/docDisplay/?docId=c04227197

HP Virtual Connect for c-Class BladeSystem Version 4.20 User Guide
http://h20628.www2.hp.com/km-ext/kmcsdirect/emr_na-c04227172-2.pdf

HP Virtual Connect Version 4.20 CLI User Guide
https://h20566.www2.hp.com/portal/site/hpsc/template.PAGE/public/kb/docDisplay/?docId=c04227190

HP Virtual Connect for c-Class BladeSystem Setup and Installation Guide Version 4.20
http://h20628.www2.hp.com/km-ext/kmcsdirect/emr_na-c04227183-2.pdf

HP Virtual Connect FlexFabric Cookbook
http://bizsupport2.austin.hp.com/bc/docs/support/SupportManual/c02616817/c02616817.pdf

HP Virtual Connect Fibre Channel Networking Scenarios Cookbook
http://bizsupport2.austin.hp.com/bc/docs/support/SupportManual/c01702940/c01702940.pdf

HP Virtual Connect Multi-Enclosure Stacking Reference Guide
http://bizsupport2.austin.hp.com/bc/docs/support/SupportManual/c02102153/c02102153.pdf

iSCSI Technologies in HP ProLiant servers using advanced network adapters, Technology Brief:
http://h20000.www2.hp.com/bc/docs/support/SupportManual/c01600624/c01600624.pdf











HP welcomes your feedback. To make comments and suggestions about product documentation, send a message to
[email protected]. Include the document title and manufacturing part number. All submissions become the
property of HP.










Get connected
hp.com/go/getconnected
Current HP driver, support, and security alerts
delivered directly to your desktop

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only
warranties for HP products and services are set forth in the express warranty statements accompanying such products and servi ces. Nothing herein
should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.
Trademark acknowledgments, if needed.
c02533991, Edition 4 - Updated May 2014