Creating an optimal network for iSCSI has many factors that must be considered during the architecture, planning, and deployment phases. This chat will connect iSCSI experts from Dell with IT professionals interested in disucssing and learning more about best practices for iSCSI networking.

Travis Vigil and Marc Farley - iSCSI Ninjas - were the expert hosts for the chat.

Technical Community – Background Reading

Virtualization Frontier blog: Why Servergeeks Should Care About Desktops

Chat Transcript

DELL-ToddM Good - lots of people actually joining on time today!
DELL-ToddM We usually get quite a few late arrivals!
DELL-ToddM Welcome hazelwood
DELL-TravisV Hello all!
Marc-Dell joined.
DELL-ToddM Today's topic is going to be iscsi Networking!
DELL-ToddM Welcome Marc!
Marc-Dell Nice to be here
mhazelwood hello
DELL-ToddM That's kind of a broad topic - and we will let the discussion go where it want to
gsherman03 joined.
slbrown joined.
DELL-ToddM We like to keep the chat as informal as possible and try to actually have some fun while talking about these tech topics
DELL-ScottH hey scott and gsherm !
DELL-ToddM So feel free to ask whatever questions you have and we will try to keep the conversation somewhat on track around our topic of the day
toratyp joined.
DELL-ToddM Hey toratyp! Welcome
DELL-ScottH welcome toratyp
DELL-TravisV iscsi Networking? It's Ip it just works :)
DELL-ToddM Many of you are veterans of previous chats - and its good to see you return today
lhundt Anyone using Md3000i with Vmware Esx?
Sean what can you say about using or 'needing' jumbo frames on iscsi?
DELL-ToddM Plug and Play right?
toratyp Good Morning from Melbourne
Sean lhundt I am.
DELL-TravisV Jumbo frames is one of my favorite topics.
DELL-ToddM Ok - Let me intro a couple of people and then we will get started on these initial questions
DELL-TravisV They are absolutely Not required.
DELL-TravisV I'm ready to go ;)
Marc-Dell right - a nice to have, but not required
DELL-ToddM Travis is an iscsi expert here at Dell and has been working in the storage area for a long time - Anything else you want to add Travis?
DELL-TravisV Nice to have means you probably won't get much better performance for most storage workloads, but you could see a decrease in Cpu utilization on your host servers. You need to make sure all the components in your San (network, Nic and array) support them
DELL-ToddM Marc joined Dell as part of the Equallogic deal and he is also an iscsi expert and leads our enterprise blog on direct2dell.com/insideit
Jeff_Glenn joined.
gsherman03 fyi they are not routable
DELL-ScottH welcome back Jeff !
DELL-ToddM When do Jumbo frames give you better performance? Any examples?
DELL-ToddM To address the Md3000i and Esx question - I have that setup in our lab and have done some testing with it
lhundt Great Sean, any problems or tweaks that you had to do improve performance? I've been pretty happy with mine but I just wanted to see if anyone else is getting good results with it?
DELL-TravisV Large block sequential workloads would benefit from Jumbo frames. Most storage workloads are small block random.
DELL-ToddM So backup jobs would benefit?
DELL-TravisV There you go
DELL-TravisV Streaming media
DELL-TravisV Back-up to disk
DELL-TravisV large file transfers
DELL-ToddM I was actually doing some experimenting with the command line interface for the Md3000i yesterday
Sean Nothing really related to the iscsi San, just tweaking resource pools in Esx mostly.
lhundt Or cloning large Vms would benefit from jumbo frames right?
DELL-TravisV Yes.
lhundt Any added benefits to using Md3000i from the command line? I mostly use the Gui interface...
DELL-ToddM From what I understand the trick with Jumbo frames is that everything has to support it and be configured to use it between the storage and the host
toratyp how do we set up jumbo frames on m1000e cisco switch 3032
DELL-ToddM And that is usually where people have problems - getting everything
Marc-Dell that's right. Nics, switches, storage controllers
DELL-TravisV I don't know offhand on the Cisco switch, but I'm sure there's a command for it in Ios :D
DELL-ToddM The command line allows more function than the Gui for the Md3000i
DELL-ToddM For example you can pull performance stats at the array and virtual disk level
toratyp I have set up storage for esx3.5 for m600 blades from md3000i for a user. they are using but do not about performance
Marc-Dell Jumbo frames often have a prerequisite for Vlans (or so I'm told)
DELL-ToddM I put an example script on a discussion thread yesterday - http://en.community.dell.com//virtualization/f/4472/p/19417969/19983709.aspx
lhundt toratyp using M600 as well with the Md3000i :)
lhundt Thanks Todd, I'll have to check it out!
DELL-ToddM I'm going to put up a wiki page dedicated to the command line stuff you can do with the Md3000i - in the next couple of days
Marc-Dell People often want to jump into a discussion about Jumbo Frames with iscsi, but it is probably more important to enable flow control
DELL-ToddM I'll post to the thread above when I get it up
DELL-TravisV Good point Marc
DELL-TravisV Especially on the Equallogic arrays
DELL-TravisV The funny thing about iscsi is that we always talk about Nics and Toes and Hbas and Cpu utilization. The ironic thing is that the vast majority of customers simply utilize Sw initiators over simple Nics and get great performance.
Marc-Dell That's true - whether its a "native" system or running as a client in a Vmware system
DELL-TravisV Good point
Sean Travis: That works for me. I've just been trying to fend off the folks here that keep confusing vm performance vs. the San's performance.
DELL-TravisV Running the initiator in the guest Os also has the added benefit of allowing direct access to all array-based Snaps and Dr/cloning capabilities.
Marc-Dell Equallogic arrays are designed for fairly high performance and you want to flow control turned on to throttle back I/os.
Marc-Dell Its a bit counter intuitive, but by throttling back I/os on congested egress ports on a switch, you improve overall performance because you don't discard nearly as many frames
Marc-Dell From time time we are asked - what's more important flow control or Jumbo frames? the answer is flow control - in almost all cases
DELL-TravisV Agreed. That's why Data Center Ethernet will be as much a boon for iscsi as it will be for anything else.
DELL-ToddM Is flow control a setting that must be enabled on all devices or just the switches?
slbrown left.
slbrown joined.
Marc-Dell Nics and switches both
DELL-ToddM Ok - I will ask the question - What is Data Center Ethernet?
DELL-TravisV It's a gleam in the Ieees eye right now. Basically they Dce is a new set of standards for Ethernet (specifically 10gbe) that will allow Ethernet to be more or less "loss-less." Today Ethernet drops packets and depends on Tcp to guarantee delivery. Dce basically makes that re-transmit from Tcp unnecessary.
DELL-TravisV left.
Marc-Dell I hate to see Travis leave mid sentence!
DELL-ScottH was travis showing an example of lossy ?
DELL-TravisV joined.
DELL-ToddM He returns!
Sean ...and thats how tcp works
DELL-ScottH travis was that your example ?
DELL-TravisV Lol
DELL-TravisV Lol
Marc-Dell ::laugh
DELL-TravisV Hey that was an unnecessary re-transmit.
DELL-ScottH ack
Marc-Dell The way I understand it Dce is a short haul, high bandwidth implementation that is non-routable - is that right?
DELL-TravisV Anyhoo, the point here is that iscsi benefits from all improvements to Ethernet such as the move to 10gbe and Dce.
Sean Fcoe!
Marc-Dell Yes?
DELL-TravisV Routing is layer 3, right?
DELL-TravisV Ethernet is layer 2 so by definition non-routable
Marc-Dell Yes, routing with Tcp is layer 3 (old age caveats apply)
Marc-Dell oops , Tcp is not routing, that's Ip
Marc-Dell old age....
DELL-ScottH @toratyp .... "system mtu" is command in Cisco Ios for setting frame ... see here -- http://www.cisco.com/en/us/docs/switches/blades/3130/software/release/12.2_40_ex/command/reference/cli3.html#wpxref20063
DELL-TravisV I'll try not to be too partisan here, but Fcoe since it doesn't have routing capability and requires Dce is short-haul and non-routable. iscsi will also run over Dce and get those same short-haul benefits, but since it still depends on Tcp/ip it's routable which is important for things like disaster recovery.
DELL-ToddM I'm curious how others are doing their networks for isci. I basically setup the same type of network that I would for fibre channel, but with ethernet switches and cables. What is everybody else doing?
Marc-Dell Another aspect of iscsi San configuration is how it is segmented from other network traffic
toratyp Thanks, Scott. I will look into it
DELL-TravisV Vlans with good Qos should be used if you share infrastructure. Logical segmentation is required. Physical segmentation is not.
Marc-Dell Physical segmentation is not required, but it has some advantages - especially for troubleshooting and network tuning. The downside is having another network to manage
DELL-TravisV Troubleshooting in that it's not as complicated. You should be able to get equivalent performance with the right network config with a logical segmentation assuming non-blocking switches of course (and massless ropes and frictionaless pulleys)
Marc-Dell Yes, the mythical non-blocking switch!
DELL-TravisV Come on, Marc. All the vendors say they are non-blocking. ;)
Marc-Dell Yes!
DELL-ToddM lhundt - You are using iscsi with M600 blades. How are you doing your iscsi network?
Lynne_smith joined.
DELL-ToddM Any additional considerations because of blades for you?
DELL-ToddM Welcome Lynn!
DELL-TravisV Which array are you using? Which Switches?
Lynne_smith I wanted to learn from the Masters!
lhundt I'm using Sw iscsi, pretty good performance...any benefits to the iscsi cards?
DELL-ScottH welcome Lynne !
DELL-ScottH use "action" .. "recent Room History" and you can catch up
DELL-TravisV I would not expect a performance improvement, but you could see lower Cpu utilization. That said, if Cpu cycles are so precious why are people virtualizing so much?
Marc-Dell I think most Intel architectures are fine with Sw iscsi. Other platforms have more difficulty
Marc-Dell I wrote Intel but meant x86
DELL-TravisV sorry Marc. Being old Dell, I austomatically assume people are running X86 with Windows or Linux.
DELL-ToddM I agree with Travis - everybody should be running x86 :-)
Marc-Dell Yeah, I mentioned it just to help people understand how Toe cards continue to exist - non x86
lhundt The virtual desktops hosted on the blades have a low Cpu performance hit so Sw is just fine, I imagine if I was running larger production servers it would be a consideration...
DELL-TravisV Are you running Vmware?
Marc-Dell its surprising how much Sw initiators in Vms can do
lhundt Yes, I'm running Vmware 3.5
DELL-ToddM We've had some discussions around the watercooler about Virtual Desktops lately - How many are you able to run per Blade server?
DELL-TravisV Are you using the initiator in Esx or in the guest Os.
lhundt 40 - 50 Virtual desktops run fine, I know I could push it to 60 or so. Each blade has Dual Quad Cores w/ 32 Gb of Ram :)
DELL-ToddM That's great! I assume that these are Xp desktops?
lhundt I'm running the Sw initiator on the Esx host itself, the vms don't even have to know anything about the iscsi stuff in the background...
DELL-TravisV Ahhh. You'll get better performance running the iscsi initiator out of the guest Os (assuming it's a Window's guest Os).
DELL-TravisV Now managing all those initiators is a different story, :) but you do have that option if you need it.
DELL-ToddM I don't think that he needs to run the iscsi in the guest - becuase they are desktops
DELL-TravisV I agree
lhundt Yes, they are Xp dekstops with 1 Gb of Ram and 1 Cpu. I haven't really stress tested each Virtual Desktop because the environment is dynamic. Students log into our lab at different times of the day and once they finish homework assignments they logoff. I work for Gainesville State College...
DELL-TravisV I'm just saying you get better performance that way. You definitely will have management challenges with all those li'l initiators...
DELL-ScottH ohhh it's a bittorrent grid :-)
Marc-Dell Scott???
MKHyde joined.
DELL-ToddM I didn't know that you could run bittorrent as a grid
DELL-ScottH yes I'm alive
DELL-ToddM Welcome Mary Kay!
Marc-Dell I was wondering what you meant
MKHyde hey!
DELL-TravisV Mary Kay!
DELL-TravisV I didn't know you liked iscsi so much...
MKHyde i live for it. dream about it
DELL-ScottH Action .. Recent Room history .... lhundt is from gainesville state college talking about students on blades
Jeff_Glenn don't you get better performance for the Guest Os if you are using Mpio.. Xp doesn't support Mpio
Marc-Dell Oh, I C
DELL-ToddM We are nearing the end of the scheduled time - mhazelwood, gsherman, others that haven't asked any questioms - Now is the time!
lhundt What's that supposed to mean, Scott...
DELL-ScottH I just mean the kids have it so easy these days .... I used to walk 10 miles uphill both directions, just to get access to a 300baud modem to download ascii art :-)
DELL-TravisV I'd have to look closer at the data. The initiator in Esx 3.5 is not high-performance Mpio capabilities aside.\
DELL-ToddM I have to drop off a few minutes early - but Scott, Marc, and Travis will carry this chat to the finish line
DELL-ToddM See everybody next week
Marc-Dell Its active passive, isn't it?
DELL-TravisV Fail-over only is my understanding
DELL-ToddM left.
lhundt Ouch, I'm lucky I never had to feel that pain, well got to head out, have a good one!
DELL-ScottH thanks for joining !
DELL-TravisV thanks for coming everyone!
mhazelwood thanks everybody
Lynne_smith thanks!
Marc-Dell I can't believe it - no stump the band questions?
Lynne_smith left.
DELL-ScottH see everyone next week
toratyp Thanks everyone
mhazelwood left.
Marc-Dell Travis, thanks a ton for joining today!
DELL-TravisV What is the air-velocity of a European Swallow?
MKHyde what?
DELL-TravisV Np! :D
Marc-Dell 2 coconut husks divided by Pi
DELL-TravisV Mk Hyde. If you don't get the reference, you may not be nerdy enough ;)
Marc-Dell but you can still like iscsi
Sean african or european?
DELL-TravisV It's a monty-python thing. All dorky teenage boys love it.
lhundt left.
DELL-TravisV Nice, Sean!
MKHyde hey travis, I am pure nerd. I dream of iscsi, remember?
DELL-TravisV Nice.
DELL-TravisV Adios everyone