In addition to virtualization, the Internet SCSI (iSCSI) protocol for storage area networks (SANs) continues to help simplify enterprise IT storage infrastructures. Recent finalization of the Fibre Channel over Ethernet (FCoE) standard offers organizations with investments in Fibre Channel SANs an opportunity to begin developing a unified network fabric for storage on a single network infrastructure. This discussion focuses on the latest developments for the iSCSI and FCoE protocols and what they mean for enhancing network storage environments for enterprises.

Technical Community - Background Reading

Chat Transcript

mike_butler FCoE?
acrotty FCoE = Fibre Channel over Ethernet
erson Are the chats going to be at 3 PM CDT all year around? You guys in Austin have switched to CST right?
Dell-ScottH This Sunday
jliebster I'm new to these. Is there an audio component to these chats?
Dell-ScottH Nope, all geeky text, well, maybe some audio J
jliebster Cool. I'll just put on my headset to look busy, then.
jerrypele I like it :)
erson /me watches Mythbusters
Dell-JeffS Hey folks, we'll get started here in a couple of minutes; working with our guest experts getting them logged in
erson They had a duct tape special a couple of weeks ago
sixth @dell-scotth, not sure if Kong relayed this to you, but I ended up going with the Dell EqualLogic PS4000V instead of the Dell PowerVault MD3000i
Dell-ScottH Have you got it yet?
sixth Nope, not yet. Estimated ship date is November 9—ordered two weeks ago :-)
erson EqualLogic PS4000XV even?
erson /me tries to imitate Ernie ;)
sixth Just need to order some switches soon...probably two PowerConnect 5420 switches
kmbishop Just interested...what was the price difference between the PowerVault MD3000i and the EqualLogic PS4000V?
Dell-JeffS Sorry, trying to help our experts in
sixth No problem; @kimbishop, it was a lot, but the benefits outweigh the cost
kmbishop =) Okay, just wondering...that’s what I figured
Dell-ScottH "A lot" is relative
sixth Very true
erson There is no EqualLogic PS4000V
kmbishop I was thinking at least double the price
erson X = Serial Attached SCSI (SAS) 10,000 rpm; XV = SAS 15,000 rpm; E = Serial ATA (SATA) 7,200 rpm
kmbishop Ah thanks, Erson
sixth Oops meant X series
Dell-ScottH Erson is fanatical about specifics :-)
erson Kmbishop, I'm guessing double the price of the PowerVault MD3000i is a good estimate
sixth Ha ha, understood
Dell-KongY How 'bout them Nittany Lions!
erson Scotth, hey, it's you that sent me the t-shirt with Ernie
sixth Yes double. I had to push hard to get it approved, but it will be worth it
Dell-ScottH Yes, because I like your "specifics" :-)
sixth Ha ha, yeah...how about that Iowa! Number 4 in the BCS! Yikes!
kmbishop I just ordered a PowerVault MD3000i, and so far it's just sitting there looking at me; haven't had to time work on it
Dell-JeffS Okay, let's get this thing going. Our experts that wrote those papers on the chat landing page will join shortly. However, I'm sure there's no shortage of opinions on this subject. I'm sure we'll have plenty to discuss until they get here. First up, the housekeeping items: this is a casual chat; feel free to jump in with your questions at any time. No need to take notes; the transcript will be posted in a couple of days. Should you see a link, I recommend you right-click it because it’s likely the interface will boot you out. If you do get booted, just rejoin and click Action, Recent Room History to catch back up
erson I think this post at the Xsigo blog is pretty good: www.xsigo.com/blog/?p=48
Dell-JeffS This week's flame bait... er, I mean, this week's topic is about FCoE, Internet SCSI (iSCSI), and Fibre Channel
jerrypele Scotth has left the building
pa Just read the title and agree
Dell-JeffS I see we have some representation from Emulex; any other storage vendors here?
Dell-ScottH Ah, back...dang Java problems
Dell-KongY What are your thoughts on iSCSI, Fibre Channel, FCoE? Are CNAs the answers?
Dell-JeffS Robert is a network expert from our CTO office, and one of the authors of the articles linked on our chat page
Dell-ScottH Author of this blog post: http://en.community.dell.com/blogs/dell_tech_center/archive/2009/10/21/the-search-for-sanity-comparing-fcoe-and-iscsi.aspx
pa FCoE is just to bridge a gap in old technology, in my opinion
Dell-JeffS Guarav, the other should be joining shortly
pa Bring on 40GbE
Dell-ScottH Pa, how many years do you think Fibre Channel has left? And 10GbE
pa 5–10
Dell-ScottH Ready for it on my laptop!
Gaurav_Chawla Thanks. I am logged in now
Dell-ScottH IMO, I'm in the same boat as you, but that's my opinion
Robert_Winter Hi everyone. Does anyone have any opinion regarding the post from Gaurav and me? Performance? Forklift? Just to be clear, Dell is in favor of FCoE—we just want everyone to have their eyes open to the challenges, but challenges make like fun, right?
erson I thought it was an excellent status report on the current state of FCoE
Dell-ScottH I think fun would be running FCoTR...TR = Token Ring :-)
Robert_Winter Thanks Erson. We tried to be fair and balanced—we're just beginning the collection of data, which we will provide going forward. We have seen some very interesting things so far...
Dell-ScottH I'm constantly confused with the Data Center Bridging (DCB), Distributed Computing Environment (DCE), and Common Event Expression (CEE) terms...who uses what and what actually constitutes FCoE?
Dell-JeffS Another post we put up recently on the subject: www.delltechcenter.com/page/comparing+performance+between+iscsi%2c+fcoe%2c+and+fc
Dell-ScottH Is it simply a "Kleenex" and "tissue" discussion?
erson I know a boy in Sweden who was just named Token Ring
erson Jeffs, I think the last graph is kinda misleading
Robert_Winter FCoE is not to be confused with DCB, DCE, and CEE. FCoE needs better Ethernet; DCB is standards-based improvements; DCE is Cisco's not-quite standard; and CEE comes from Brocade—not quite standard
jerrypele Token Ring sounds like a Hobbit :)
Dell-JeffS That last link takes on the "low-performing, lossy, and unpredictable" myths about iSCSI
Robert_Winter iSCSI can work over DCB just as well with great benefit
Dell-KongY To the community members, does the underlying infrastructure even matter as long as the end user gets satisfactory QoS? Or do you have so much vested that you need to go down a certain path?
erson Not really sure how those graphs on 512 KB got so high for iSCSI when the corresponding percentage data points were pretty much identical
Dell-ScottH So it's similar to when I upgraded my home wireless to faster speeds, but the standards weren't ratified yet? So is DCB ratified and blessed? If so, when can we start expecting products built on DCB?
Robert_Winter I think you may be looking at the efficiency data points for iSCSI. DCB, 802.1qbb and 802.1qaz will be ratified early next year; 802.1qau not for another year. There are already DCB products, but they *almost* don't work...not very stable yet
erson Is it for certain that they will be able to fully support the finished standard when it is released?
Robert_Winter Yes, the vendors are following very closely. There is some "slop" in the standard such as how many priority queues you support
Dell-ScottH Just found a decent presentation from Storage Networking Industry Association (SNIA) on "Ethernet Enhancements for Storage": www.snia.org/education/tutorials/2009/fall/networking/SunilAhluwalia_Ethernet_Enhancements_for_Storage.pdf
erson How will DCB help iSCSI?
Dell-ScottH Oh, and just found all the gorpy details: www.ieee802.org/1/pages/dcbridges.html
Gaurav_Chawla Erson, regarding your question on performance at 512 KB, the difference in CPU utilization is not visible in the charts because of the size of the chart. The % CPU numbers for 512 KB reads were 0.76 for iSCSI, 1.23 for FCoE, and 1.16 for Fibre Channel. For 512 KB writes, the % CPU numbers are 0.67 for iSCSI, 1.11 for FCoE, and 0.99 for Fibre Channel. This is the reason efficiency for iSCSI is higher
Robert_Winter DCB will keep the number of TCP re-transmits down and balance the load from multiple initiators—we've seen this behavior in the lab
erson And can we assume that DCB will be a standard on every enterprise switch a couple of years from now?
Robert_Winter No, I personally do not think DCB will be on every switch; but Ethernet is diverging into different vertical technologies, server provider, AV, etc. This means many types of switches are not all interoperable—in short, a mess
Dell-JeffS I saw an interesting comment on Twitter about a new card from Emulex today
erson Gaurav, thanks for the detailed data. I knew that the explanation would be hiding in those small numbers. But when the % CPU is so low for all three protocols the efficiency diagram really didn't tell me anything since the actual difference in CPU utilization percentage was negligible
Dell-JeffS A 10GbE that did it all, just license what you want. iSCSI, FCoE...any thoughts, @emulexgirl?
erson OneConnect Universal Converged Network Adapters (UCNAs)?
Gaurav_Chawla Erson, if a server has multiple 10GbE ports, then CPU utilization numbers will start to grow. But you are right that CPU is still low for large I/O sizes
Dell-JeffS @erson, I *think* that is what the tweeter was talking about
datanow Thanks for joining Brent. Emulex has just announced today a UCNA card that in time will support 10GbE along with iSCSI and FCoE, all hardware offloaded
erson Guarav, yes, you are right. That would of course make a larger impact. I'm not sure when servers start to use more than two iSCSI 10GbE though
Robert_Winter Assume that DCB is not 100 percent lossless; what happens when you drop an FCoE packet? There's no TCP to fall back on; SCSI-level timeouts? 60 seconds?
Robert_Winter It has not been proven that DCB is lossless—only stated. FCoE isn't Fibre Channel. It's just a marriage of convenience between the Fibre Channel protocol and a better Ethernet, but not a perfect Ethernet
erson Robert, what about 40/100GbE...will FCoE work right away with those, or are these standards that we are waiting purely for 10GbE FCoE?
Robert_Winter @erson, DCB pause is link speed specific so you have to make sure that your pause delays are designed appropriately; Fibre Channel credit-based pause is speed agnostic, so you may have some issues with 40GbE. DCB pause is the same Pause frame technique as 802.3x, just replicated 8 times. 802.3x Pause didn't work that well
erson I'm guessing the actual gain from using iSCSI on switches with DCB/DCE/CEE support is slim to none?
Dell-JeffS We've spent a lot of time on the FCoE side of things...anything new you guys learned in your testing about iSCSI? I think the common misconception (which I held as well up until joining Dell and actually using it) is that it isn't enterprise ready
Robert_Winter @erson, no the gain is real and significant. We can't release our data yet, but as I said it "evens" out the iSCSI flow because of reduced TCP re-transmits and balances multiple sources into a target benefits, but iSCSI works without DCB while FCoE can't
erson Looks like you've covered performance between iSCSI, Fibre Channel, and FCoE pretty well. I would like to see some actual cost comparisons between the three
Gaurav_Chawla In addition to the priority flow control (PFC) feature of DCB, DCB also provides additional features that will allow users to reserve a certain percentage of network bandwidth for iSCSI traffic
Gaurav_Chawla This gives QoS for iSCSI for end-to-end flow
Robert_Winter @erson, how much money do you want to spend?
erson Were there any URLs to a good primer on DCB posted above?
Gaurav_Chawla Without releasing actual cost numbers, we can safely say that iSCSI provides cost flexibility. You can use iSCSI in SW or iSCSI over TCP Offload Engine (TOE) or full iSCSI offload
Robert_Winter @erson, ieee.org has a lot of public information on its site, but this is probably something we need to post somewhere as Dell because we do have that content, but probably not publicly accessible—we'll check into that
erson Robert, it wasn't a particular question from my point of view (I'm hook, line, and sinker in the iSCSI camp), but I'm guessing that Dell customers want to know more about the actual cost difference
Gaurav_Chawla On the Ethernet side, you have flexibility with iSCSI, because you can use existing 10GbE switches, 1 GbE switches, and upcoming DCB switches. The whole network doesn't have to be DCB capable. With FCoE you need a DCB-capable network. For FCoE you also need new FCoE-capable converged network adapters (CNAs), and you need FCoE forwarder (FCF) to bridge to existing Fibre Channel storage area network (SAN)
erson Yeah, and the number of DCB pre-standard-compatible switches today are the Nexus line. Any others?
Robert_Winter @erson, currently I think it's safe to say that you will pay a premium for DCB and FCoE because they're new gadgets
erson To me it seems that FCoE right away won't bridge the cost gap between Fibre Channel and iSCSI in any way
Robert_Winter @erson, there are a couple more switches that are DCB capable; I don't like any of them so far
Gaurav_Chawla Erson, Brocade being a Fibre Channel and Ethernet player will have DCB and FCoE switches. and I am sure other 10GbE switch vendors will offer DCB capabilities as well
Robert_Winter @erson, I think you're correct. FCoE requires a whole new infrastructure, which is additional cost that iSCSI doesn't require
erson BTW, that performance comparison earlier...the iSCSI was used with iSCSI Offload Engine (ISOE). Was it a Broadcom 57711 NIC?
datanow Does Extreme support DCB?
Gaurav_Chawla Erson, we ran all the three protocols (iSCSI, FCoE, and Fibre Channel) across a broad spectrum of initiator vendors. And we took the best performance numbers. So it is not limited to Broadcom
Robert_Winter @datanow, you'll have to ask Extreme. All switch vendors have plans; some are closer than others. Extreme is very active in the standards forums
erson Are the Broadcom with ISOE switches comparable in that they manage to offload to the "real" iSCSI HBAs?
Gaurav_Chawla Yes, it is very much comparable and offers very good, competitive performance
erson The HBAs seems to have some more features regarding boot from iSCSI, but regarding offload they seem very comparable
Gaurav_Chawla Boot from iSCSI is also offered for SW iSCSI initiator. OS and NIC handshake through a standard called iSCSI boot firmware table (iBFT)
erson I hope…doh. I hope we can continue this discussion during the time that FCoE matures
Robert_Winter FCoE: Gaurav and I are intimately tied together until it gets better...
erson Gaurav, yes, it sure does. I tried to help Jeffs with some input on what at least I would like to see in the new solution guide for BFI.
erson 7_of_9 was a hot nick :)
Gaurav_Chawla Erson, great. Thanks! It has been great chatting with you
erson The same to you
Dell-JeffS Okay, it's that time. Robert and Guarev have other commitments to get too. Thanks Robert and Guarev for joining us today!
Dell-JeffS Thanks everyone for joining; look for a transcript in the next day or two
datanow Thanks Jeffs
erson I felt like I was nearly the only guy chatting? Is this chat working correctly?
jerrypele You and Robert
Dell-JeffS I thought it would have been more back and forth. Still a new technology I suppose. Glad you were there to drive the discussion!
erson Okay, good I didn't miss anything
datanow Good discussion. I did not have any solid performance numbers to share
erson I do think it was a very good and interesting topic
Dell-JeffS Okay folks, next week Scotth will be sharing some of his recent work with Repository Manager. See you then!
jerrypele Okay, bye
erson It was really good to have those two blog posts to lean on
Dell-JeffS Very true. Have a good one!
erson Okay guys, I'm out