Comparing Performance Between iSCSI, FCoE, and FC

Storage

Storage
Gain insight and information on Dell Storage products from engineers, peers and experts in the industry.

Storage - Wiki

Comparing Performance Between iSCSI, FCoE, and FC

Storage - Wiki

Comparing Performance Between iSCSI, FCoE, and FC


The following is written by Ujjwal Rajbhandari, from Dell Storage Product Marketing Group.
433845

There are number of discussions, blogs, and articles comparing Internet SCSI (iSCSI), Fibre Channel over Ethernet (FCoE), and Fibre Channel (FC). Many of them share a common belief that FCoE and FC are better suited as core data center storage area networks (SANs) and that iSCSI is ideal for Tier 2 storage or for SAN deployments in remote or branch office (ROBO) and small and medium business (SMB) environments. That is because iSCSI is characterized as “low-performing,” “lousy,” and “unpredictable.” In this blog I will tackle the misinformation around iSCSI performance as compared to FC and FCoE. I will also compare effective efficiency of the various SAN protocols since efficiency is an aspect of performance.

Both iSCSI and FCoE share the same 10 Gigabit Ethernet (10GbE) at the transport layer. However, the perception is that the TCP/IP overhead makes iSCSI inefficient compared to FCoE and FC (both having better payload to packet-size ratio), thus leading to lower performance and efficiency. Figure 1 shows protocol efficiency calculation for iSCSI (both 1.5K MTU and 9K MTU), FC, and FCoE (2.5K MTU). It can be seen that when jumbo frames are enabled, iSCSI has the best protocol efficiency.

image

Figure 1: Protocol efficiency comparisons


Regarding performance, iSCSI having low performance might have been true when 1 Gbps was the maximum throughput available per iSCSI port (whereas FC was delivering 2 Gbps, 4 Gbps, and 8 Gbps per port), but with the availability of 10GbE, the commonly held belief that iSCSI performance is not up to par compared to FCoE or FC is no longer true.

The Office of the CTO at Dell conducted a series of performance tests to compare 10GbE iSCSI, FCoE, and 4 Gb FC. To ensure similar workloads, the application throughput was limited to 4 Gb. The host bus adapters (HBAs) used for the different protocols were as follows: a 10GbE network interface card (NIC) with iSCSI offload for iSCSI traffic; a 10GbE converged network adapter (CNA) for FCoE traffic; and a 4 Gbps FC HBA for Fibre Channel traffic.

The goal of the testing was to capture achieved throughput and CPU utilization for a given SAN protocol.

The protocol efficiency comparisons from Figure 1 might be theoretical in nature; Figure 2 shows results from an I/O workload study comparing throughput of 10GbE iSCSI, FCoE, and 4 Gb FC HBAs. To keep the results easy to visualize, the results show the throughput achieved when the application generated 4 Gb throughput. It can be seen clearly that iSCSI outperforms FCoE and FC regardless of read or write operations for various I/O block sizes.

image

Figure 2: Throughput performance comparisons (MB/s)


Along with capturing the throughput, let’s examine the host CPU utilization to better assess the performance and efficiency of specific SAN protocols. All the host adapters are comprised of hardware-based offload capability to process the protocol-specific traffic, minimizing use of CPU resources. Figure 3 shows the effective CPU utilization for various workloads. It can be seen from this figure that all the host adapters have similar CPU utilization metrics, again reinforcing the fact that iSCSI is as efficient as FCoE and FC.

Finally, Figure 4 shows throughput efficiency, defined as MBps/%CPU, for the various storage protocols. The chart shows 10GbE iSCSI having the best throughput efficiency across the workload types, clearly outperforming FCoE and FC.

From the test results we can undoubtedly summarize that iSCSI as a SAN protocol is not “lower-performing” or “inefficient” compared to FC or FCoE. On the contrary, iSCSI outperforms both FC and FCoE. Customers who are planning to purchase storage for their data centers can consider an iSCSI SAN as a viable option, knowing iSCSI performance is at par or even better than FCoE and FC. Also, customers considering unifying their data center networks over Ethernet can start doing so now with iSCSI. While FCoE can also deliver storage traffic over Ethernet, it is still under development and is not ready for prime time.

image

Figure 3: CPU utilization (%) for iSCSI offload, FCoE, and FC


image

Figure 4: Overall protocol throughput efficiency (MBps/%CPU) for iSCSI offload, FCoE, and FC
Comments
  • Why would you compare FCoE and iSCSI on a 10Gbit network interface vs. Fibre Channel running at only 4Gb?

  • @davidsinaus:  You will notice that the bandwidth for FCoE and iSCSI were managed to 4Gb via ETS to ensure that it was an apples to apples comparison in terms of bandwidth.  Sicne DCB - a requirement for FCoE - requires a DCB capable CNA and switch, which are all no "slower" than 10Gb, you have no choice ethernet bandwidth class of devices.

  • You compare a 10Gb iSCSI network with a 4Gb FC network, but for the price of a 10Gb iSCSI network you can buy a 8Gb FC network. The price is the only common element you can compare. Which performance you can have for the same price.

  • @ManuFr Have you ever bought a FC switch? If you look at Brocade based switches, it´s not only the FC switch itselft you have to buy, add the fc modules, add licenses, add software and hardware support contract for the hardware. You can get 10Gbe ethernet switches for a much lower price nowadays...

  • Now that 2 years have passed, do you still agree with this compelling attempt to revive iSCSI?

    iSCSI on Jumbo frames must ensure that every device from Initiator to Target has Jumbo frames enabled as well as identify if the Target can handle the 9k payload.  

    After reading the initial 2 paragraphs I cannot tell if the iSCSI comparison with FC/FCOE is based off of the Jumbo frame or 1500 MTU frame. I am guessing Jumbo, is that correct?

    What is the chance of comparing FCoE on Jumbo Frames to iSCSI?

    No matter how you dice it, FC is the most reliable method.