This discussion focuses on 10 Gigabit Ethernet options on the Dell™ PowerEdge™ M1000e blade enclosure.

Technical Community - Background Reading

Chat Transcript

DELL-ScottH Welcome everyone
VeeamMark Are we going to have a live Dell TechCenter chat from VMworld?
Dell-KongY Yes, he will be traveling with us :)
VeeamMark No boat this time...just plenty of party
Dell-KongY That's a possibility...maybe a tweet up and you are most definitely invited
DELL-ScottH Yes, we will be tweeting, chatting, and blogging live from VMworld
ceri Bah, I have declared myself too busy for VMworld even :(
Dell-KongY @ceri, sorry to hear that :(
DELL-ScottH Love to be busy, but today has been off the hook for me—back to back to back
ceri Hope you all have a great time, honest ;-)
Dell-KongY Welcome, Ujjwal
DELL-ScottH We will bring back some schwag for you...and take pics :-)
ceri I'll hold you to that, thanks!
VeeamMark I am excited because I have a full pass for the first time ever
Dell-KongY BTW, Ujjwal is from our storage marketing arm; but before that he was an uber Oracle/SQL/Linux engineer :)
DELL-ScottH Oh man, you in sessions, can't wait for that
DELL-ScottH welcome Ujjwal !
VeeamMark What is today's topic?
DELL-ScottH Today's topic is 10GbE options on Dell PowerEdge M1000e blades
Dell-KongY Welcome Ken! Welcome Erson!
erson Hi all
Ujjwal_Rajbhandari Back again, lost my connection…
DELL-ScottH There's Erson; we were waiting for you :-)
erson Just got back from a restaurant; /me leans back in his chair
DELL-ScottH Okay, let me tweet the start one last time, and then we'll kick it off
DELL-ScottH No belching :-)
VeeamMark Thank goodness I have my sound off
DELL-ScottH Okay, welcome everyone to this week’s chat session
erson Great reading suggested for this chat
DELL-ScottH Today's topic is 10GbE options on the PowerEdge M1000e blade chassis—aka "unified Fabric" :-)
DELL-ScottH More to come on that later this year. Okay, a couple of reminders. The chat is recorded and you can view the transcript tomorrow or any time later from the same link that brought you here. Send it to your friends. Feel free to ask questions at any time; you aren't being rude by interrupting. It looks like a lighter audience this week, so there should be some good discussion. Today it is just the TechCenter team and a few other Dell employees (say hi)...so we'll keep it mean and clean and to the techno-extreme :-)
DELL-ScottH The first document that will be a great help in getting you started is this Blades I/O guide: www.dell.com/downloads/global/products/pwcnt/en/blades-io-guide-2009.pdf
DELL-ScottH That will show you all the currently available I/O options for the chassis, specifically in the 10 Gb space we have three current switch offerings
ceri That link alone was worth attending for
DELL-ScottH Two PowerConnect solutions and one Cisco. One the first PowerConnect side is the M6220, which offers 1 Gb internally, but has 10 Gb options out the back
VeeamMark Scott, this may be off topic; are you doing anything with Xsigo?
DELL-ScottH @ceri, bookmark it :-)
DELL-ScottH Yes, funny you should mention that...let me speak to that in a second
VeeamMark Sweet
ceri @scotth, I have saved a copy locally ;-)
DELL-ScottH Details on the M6220 are here: www.dell.com/us/en/enterprise/enterprise/pwcnt_6220/pd.aspx?refid=pwcnt_6220&s=biz&cs=555
DELL-ScottH It gives you 16 internal 1 Gb and then two slots for optional uplinks. Page 4 of the blade I/O document referenced earlier gives the full details
JasonPowell /me is back...yup, had the M3 autoxing last Saturday :-)
DELL-ScottH You have 10 Gb connection options for copper (10GBase-T), copper (CX-4), Fibre XFP and Fibre SFP+...the most connectivity options
DELL-ScottH @jason, very sweet :-)
erson The performance of the PowerEdge M1000e is impressive in regard to Fabric B and C, 40 Gb/port is a lot and should be enough until the chassis is replaced
DELL-ScottH The second PowerConnect option is the M8024
DELL-ScottH @erson, agreed
DELL-ScottH Details on the M8024 are here: www.dell.com/us/en/enterprise/enterprise/switch-powerconnect-m8024/pd.aspx?refid=switch-powerconnect-m8024&s=biz&cs=555
DELL-ScottH This switch gives you full 10 Gb connectivity inside and out
ceri This is the only switch that puts 10 Gb to the blades, correct?
DELL-ScottH It connects to B or C switch module bays and has 16 internal 10 Gb ports and external options for CX-4, 10GBase-T and SFP+
DELL-ScottH @ceri, correct—currently
ceri Don't suppose it's really a Cisco or a Brocade in disguise? ;-) I have to sell it to our network guys
erson My guess is that it's not, considering the price
VeeamMark Ah, the old is-PowerConnect-actually-Cisco question :)
ceri Didn't think so ;-)
DELL-ScottH The third option available is the Cisco 3130x
VeeamMark <-- Old Dell PowerConnect engineer...that brought tears to my eyes
DELL-ScottH @ceri, nope. Just get the Cisco switch if your network guys are being buggers :-) That's why we have lots of options :-)
erson I was quoted just above $3,000 for a M8024
DELL-ScottH The details on the Cisco 3130x are here: www.dell.com/us/en/enterprise/enterprise/cisco_3130x/pd.aspx?refid=cisco_3130x&s=biz&cs=555
DELL-ScottH Wow, that is cheap enough for my house rig :-)
ceri @erson, doesn't quite translate to the equivalent in £, but I have a quote for < £2300
erson Anyone have an idea what a reasonable price quote for those Cisco switches would be?
DELL-ScottH Alright people, lets keep it technical here, before I give you the slap down :-)
ceri Lol
DELL-ScottH It's all free to me in the lab :-)...beat that!
ceri Okay, so how many internal ports on the Cisco?
DELL-ScottH So getting back to the Cisco (thanks, Ceri :-)
DELL-ScottH 16 internal ports 1 Gb
DELL-ScottH With four external 1 Gb and two slots for the 10 Gb options...similar to what I explained for the M6220
erson Hmmm, blades-I/O-guide-2009.pdf is missing some information regarding the M8024 on page seven. There is no information about the SFP+ DA (direct attach) cables
ceri (@erson, I can say that the quote I have for the 1 Gb Cisco 3032 is £4 more than the M8024...)
erson Every switch has 16 internal ports (all have 1GbE per internal port except M8024, which have 10GbE). Do we have an expert on the midplane architecture here?
DELL-ScottH @erson, yes that is a great summation. Midplane?
erson Yes, the passive board in the middle
erson It's called the midplane...at least in Dell’s documentation :)
DELL-ScottH I don't think we have any midplane experts on the line, but what is the question, and maybe we can ping somebody on internal IM. Also, getting back around to Mark’s question about Xsigo, we have one in the lab and have tested it on vSphere. I also did a quick demo on it
AlanBarber Side question, do any of the EqualLogic arrays support 10GbE?
DELL-ScottH Demo for Xsigo (don't get sick watching me spin around): www.delltechcenter.com/page/xsigo
erson Like I said above, I've always been impressed by the performance of Fabric B and C (each mezzanine card has two ports and each port can potentially use 40 Gb/sec.), but I was looking into the possibilities of Fabric A yesterday and had some questions regarding its possibilities. How much was that Xsigo?
DELL-ScottH Free to me :-)
erson I remember scotth said something about a price one time
DELL-ScottH Xsigo is more about adding up the price of everything and comparing
shanfont Hey guys. am I too late?
DELL-ScottH Not really fair to look at switch to switch, because they aren’t equivalent in that nature
DELL-ScottH Hey Shane! How was Austin last week? You can catch up using Action, Recent Room History
shanfont All work no play; updated my ESX servers to Update 4
DELL-ScottH @erson, there are some interesting possibilities for slot A in the future (mid next year maybe)
VeeamMark So, Scott, the reason I asked about Xsigo is that they mentioned there is the possibility of increasing backup performance, which as you know would be of major interest to me
erson Anyway, regarding Fabric A: apparently it has four lanes for each half-height blade. And each lane seems to be able to carry 1GbE, thus a total of 4GbE. Then why don't the half-height blades have four NICs (at least the new M610)?
DELL-ScottH So Xsigo uses HCAS (Infiniband) and can do 20 Gb bidirectional...so up to 40 out per server
VeeamMark Interesting
ceri @erson, are they bidirectional lanes?
erson I'm assuming a problem is that the current GbE-passthrough and all the GbE switches only have 16 internal ports—16 blades using 4 NICs wouldn't really work
erson Each lane is bidirectional
ceri Okay
Dell-d_glynn @alan, not currently
erson So it's capable of 4GbE NICs. Fabric A is dedicated to Gigabit Ethernet. Though initial server module releases are designed as dual Gigabit Ethernet LOM controllers on the server module planar; the midplane is enabled to support up to four Gigabit Ethernet links per server module on fabric A. Potential data bandwidth for fabric A is 4 Gb/sec. per half-height server module.
erson Copy paste from www.dell.com/downloads/global/products/pedge/en/pedge_m1000e_white_paper.pdf
ceri I assume that the answer there would be that most customers don't need it. Seems to be the answer every time I'm being awkward anyway :-)
erson And what about 10GbE LOMs...they should for sure appear before this chassis is EOL
DELL-ScottH @erson, that's basically what I'm alluding to
erson Hey, PowerEdge M610 and PowerEdge M710 are kings of virtualization...of course NICs are needed. Scotth, I need to stir up some 12G rumors that you can confirm then :)
Dell-JeffS @alan, EqualLogic plans support later this year, subject to change of course
DELL-ScottH Ah, speaking of that: www.delltechcenter.com/page/12g+sneak+peek
Dell-KongY Enjoy the 12G sneak peek
erson Doesn't work for me sadly
Dell-KongY You are bound by the secret TechCenter oath
ceri What's 12G? (I am a Dell newbie, obviously)
DELL-ScottH @erson, no way! I was thinking of you when I made the page :-)
erson But I'm guessing it's a Rickroll or similar (yes, I got to Youtube, but it denied me access because of my originating country)
erson ceri, 12th- generation servers
DELL-ScottH Our servers are enumerated by generation: 1st, 2nd, 3rd, etc. We are on the 11th
ceri Ah ha, last 11th-generation being?
DELL-ScottH PowerEdge 2850 was 8th, PowerEdge 2950 was 9th, and then of course we changed names and broke the scheme :-)
erson I'm also assuming that the current dual-port 10GbE mezzanine NIC is using 10GBase-KX4, correct? Or could it be that you’re already using 10GBase-KR?
Dell-KongY 11th-generation = PowerEdge M610, PowerEdge M710, PowerEdge R610/R710
ceri Explains a lot. Most recent Dell kit we have is PowerEdge 1950s
erson And the T’s...
DELL-ScottH Ah, time for an upgrade
VeeamMark Has Rickroll been officially entered into the dictionary now?
Dell-KongY @erson, yes the towers as well :)
DELL-ScottH @erson, let me check on the KX, KR...was just on a call about that and some of the futures
ceri We thought we had upgraded; just a different vendor;-p
DELL-ScottH @mark, it is for sure in wikipedia
DELL-ScottH Doh! Time to come back :-)
VeeamMark Scott, I heard from VMware that it released support for 12-core processors; any word on when Dell will be putting out such beasts?
DELL-ScottH Hmmm, we don’t make processors
VeeamMark I didn't say you had to make them
erson Veeammark, within the end of this year
VeeamMark You just need to support them in your systems :)
ceri @veeammark, I think they just have their licensing road map sorted
DELL-ScottH VMware is just getting ahead of the curve as always
erson Intel will release their 8-core Nehalem-EX this year (16 logical processors with their new SMT [aka, hyperthreading 2.0])
DELL-ScottH As an official spokesperson for Dell, I can neither confirm nor deny such rumors at this time; hit me up when it's not recorded :-)
erson Two to four sockets will be a common configuration with some beasts (not from Dell, though, I'm guessing); will probably go up to eight sockets—16 DIMM slots per socket
VeeamMark You have a future in politics
erson FB-DIMM2
ceri @erson, licensing is on physical cores—that's why I think it's just a licensing thing. It's also "12 cores per package," not per system
DELL-ScottH I let Erson leak everything...I have no idea where he finds this stuff :-)
erson An eight-sockets system could potentially have 2 TB of memory. I haven't leaked anything about Dell’s stuff...at least not yet... :)
DELL-ScottH 2 TB and eight sockets is most definitely a RISC killer
ceri Anyway, if I may talk about 10GbE for a second ;-)
erson My synapses are connected to Google...that’s how I find my information
DELL-ScottH Ah, yes
VeeamMark Yes, 10GbE
DELL-ScottH Question on 10GbE, please
ceri Assume I have PowerEdge M610 servers and the PowerConnect M8024—no way to make that redundant across switches (as there are only 16 internal ports)? Or can I have two PowerConnect M8024 switches?
erson Sorry, got a bit carried away there :)
DELL-ScottH You put two switches in the back bays
erson Just a tip for the latest in Intel/AMD tech: http://pc.watch.impress.co.jp/docs/column/kaigai/
ceri Obviously, I only want one 10GbE mezzanine, as I need the other for Fibre Channel; that's doable?
DELL-ScottH On each blade you have a B and C mezzanine card option...
ceri So two switches on Fabric B, 10GbE card in mezzanine slot B, sorted
DELL-ScottH A is always routed to what is on the planar; B routes to B and C to C. So if you put 10GbE in B you put the PowerConnect M8024 in B in the rear, and then you put Fibre Channel in C and route to C in the rear with Fibre Channel switch or pass through...get it?
erson One recommendation about blade servers when it comes to redundancy. Only the full-height blades have four slots for mezzanine cards. That means they have two mezzanine slots for Fabric B and two for Fabric C—aka, mezzanine redundancy. Take for example the PowerEdge M710
ceri Yep, thanks
DELL-ScottH Look at page 2 of that I/O guide
ceri Oh, you're asking me to stop doing four other things, just a sec...
DELL-ScottH Lol
DELL-ScottH Yeah, I just loved when I picked up the twitter bug, all the other things didn’t stop—focus, focus, focus! Inside joke :-)
ceri Okay, I am looking at page 7, the PowerConnectM8024 page. I'm noting that it says, "Broadcom 57710 dual-port 10 Gb..." and getting it ;-)
VeeamMark Scott, monitoring hardware on the blade enclosures...do the switches send our SNMP or IPMI alerts?
ceri Also, importantly, are they configurable through SNMP?
DELL-ScottH Yes, the switch modules communicate with the chassis management controller (CMC) and send out SNMP. The switches are also "real" switches
VeeamMark Do you know if they will be moving to IPMI or CIM/Smash anytime soon?
erson IEEE is working on the standard for 40/100GbE. The PowerEdge M1000e could possibly get dual-port 40 Gb mezzanine cards
DELL-ScottH With respect to the same full-featured management features that are built into every PowerConnect switch, or Cisco switch, the Cisco runs the same Cisco I/Os as every other Cisco switch, so give it an IP and let the network guys do as they will with it
ceri Yeah, I need the PowerConnect M8024 though
DELL-ScottH @mark, not sure about the switches, but Smash is in the latest firmware of the CMC
erson Could you confirm that SFP+ DA is an option for short range connection with the SFP+ modules on the PowerConnect M8024? It should be so that we can forget CX-4 and all the other options
DELL-ScottH @ceri, all the PowerConnect M8024 documents are here to browse: http://support.dell.com/support/edocs/systems/pem/en/index.htm
erson If so, we could use SFP+ for both short-range copper connections (works exactly the same as CX-4, so it's really low latency) and with transceivers for medium-long range Fibre Channel connections
ceri @scotth, thanks, that will be useful
erson Not needing a transceiver for SFP+ DA makes it also a cheap solution like CX-4
DELL-ScottH @erson, I'll check on the SFP+ DA and get back to you
DELL-ScottH I would like to thank everyone for joining today. The transcript will be available for browsing tomorrow, so don't worry about writing down URL links. Any parting thoughts or questions?
Dell-KongY Thank you, Scott, for hosting and moderating the chat
DELL-ScottH Time to send out a chime to wake Jason up :-)
JasonPowell /me, what the…
DELL-ScottH Ha ha
JasonPowell Freaked me out…/me turns down speakers
DELL-ScottH I know I can always get you back in the chat with that! :-)
JasonPowell :-)
shanfont Yeah, I had my speaks loud
DELL-ScottH Thanks again for joining. If ya'll have any questions, just post them on the site
VeeamMark Man, my ears hurt
ceri So, what's next up?
DELL-ScottH So, Mark, what are the details on the Veeam party?
VeeamMark Tuesday night
DELL-ScottH I think Audi R8 is next :-)
DELL-ScottH Oh, next chat?
JasonPowell Drool
VeeamMark Some establishment that has activities to your liking
ceri Heh ;)
erson Ceri, next time it's SBUU 6.1
Dell-KongY Yep, that's Scott's next automobile
DELL-ScottH Yeah, Systems Build and Update Utility
ceri Oh, looks good. Bare metal, naturally?
JasonPowell I've been drooling over the new LS1 swaps in a Miata—even LS6!
DELL-ScottH I cannot get the R8 out of my head; such a sweet car, and just about in my reach...if I sell everything!
VeeamMark Better flog some more torpedoes
JasonPowell Our PowerEdge R710s arrive Thursday or Friday!
erson A car is something that gets me from A to B; that’s pretty much what I know about cars
DELL-ScottH Yes, but I like to go to A', A'', and B""" :-)
DELL-ScottH @jason, Whatcha doing with the PowerEdge R710 servers?
erson /me should really accept those chat pop-ups and talk with a Dell sales consultant and terrorize them with hard questions, besides caressing them
DELL-ScottH Be nice :-) They are usually fresh out of school :-)
JasonPowell We're moving all VMware server VMs to the PowerEdge R710 servers running vSphere
erson Scotth, would be nice if you could get that 10GBase-KX4 and 10GBase-KR information as well
JasonPowell And moving from ‘03 domain to ‘08
DELL-ScottH Have fun with that; all your planning and prep in place?
erson Scotth, just found this on the tech specs for PowerConnect 6248 (well, I knew about it before but had to make sure)
JasonPowell Prep?
erson Cables, SFP+ direct connect cables; the PowerConnect 6200 series has dual-slot SFP+ modules
VeeamMark Scott, why would you want to plan and prep for an upgrade?
erson And if they have SFP+ DA cables so should the PowerConnect M8024
shanfont Planning and prep? When does that ever go right?
VeeamMark Failing to prepare is like preparing to fail
erson You should really push that SFP+ can replace CX-4 besides also being used with transceivers for medium-long range Fibre Channel connections
JasonPowell Actually, we've started to whiteboard it a little. But we'll soon fill up the 4x12' whiteboard with deets
DELL-ScottH I knew it—you guys tell the VPs everything is good to go, but you are just flying by the seat of your pants :-)
erson So we can get rid of all other 10GbE interfaces beside SFP+ (I'm assuming that 10GBase-T will stick around)
shanfont That's the way I do it.
JasonPowell Sadly, we're low on SAN space so converting from VMware to vSphere is going to be "interesting"
shanfont Good luck, Jason
DELL-ScottH Yes, good luck. I expect many tweets, and hopefully not late at night
shanfont See you guys next time, if I don't forget
DELL-ScottH Later, Shane, thanks for coming by
VeeamMark Jason, you can use Veeam to reclaim some of your storage LUNs dedicated to backup, unless you are already using a good synthetic-based approach :)
JasonPowell David Siles did a Veeam demo for us...sweet stuff
erson Scotth, can the Xsigo be used with blade servers? You need an Infiniband connection from every server to the Xsigo. Do any Infiniband NICs do, or are Xsigo’s own Infiniband NICs" needed?
JasonPowell No budget :(
erson Veeam for Hyper-V soon I hope?
VeeamMark Oh, well if you have already spoken to the master, my work here is done :)
JasonPowell Lol
VeeamMark Erson, yes. Are you on the beta interest list?
erson Veeammark, within a week or two? Nope, but would for sure like to be
DELL-ScottH Can use it on the blades, supported with our Infiniband adapters on the blades
VeeamMark Reach out on our forums
DELL-ScottH Just needs the Xsigo driver on top
VeeamMark Anton Gostev is the guy to get in touch with
erson Okay, are there hints on what you are going to announce for Hyper-V anywhere?
VeeamMark Jason, don't let budget stop you. We can work something out
erson This is an image I like: http://pc.watch.impress.co.jp/img/pcw/docs/304/653/html/kaigai15.jpg.html
VeeamMark There are a couple Easter eggs in the forum, but your best bet is reaching out to Anton on our forum :)
erson I'm thinking that Dell will probably just make a refresh off their current Nehalem servers to support that hexacore-replacement that arrives 1h' 2010
erson As in hardware is “very” nice addition to the hexacore indeed. Who said something about the 12 cores limit on vSphere?
VeeamMark I mentioned that VMware is now supporting 12-core architecture
erson Is it because AMD has announced that they will release a dual-die 12-core processor possibly in 2010? “Magny-cours” is the codename. /me is thinking that a PowerEdge M710 should be outfitted with redundant dual-port 10GbE mezzanine cards on Fabric B to support redundant two 10GbE connections to the dedicated iSCSI switch stack and redundant two 10GbE connected to the internal/server network, and then use the four NICs on Fabric A for management, backup, cluster network, or whatever that seems suitable. DMC second look on 11th of August...my last day on summer vacation :)
Dell-d_glynn @erson, you’re pretty up on Microsoft and Hyper-V; any advise for the test 70-652?
erson Get the Microsoft Press book, it's excellent. I intend to take that exam later this year hopefully. I've heard that the exam is pretty easy, though, so if you're familiar with the general usage of Hyper-V and with SCVMM 2008 you shouldn't have any problems with the exam
Dell-d_glynn I'm not 100 percent on all the Microsoft terms; I tend to default to the more “industry” terms for things rather then Microsoft’s. SCVMM, I've only touched a few times. Maybe I should just a jump in and take the exam
erson Dell really lacks a PowerConnect 10GbE switch, and it wouldn't be bad if the series replacing the PowerConnect 6200 gets the same modules as the PowerConnect M8024, so you get up to 48 GbE ports and up to 8 10GbE ports (SFP+)
erson Scotth, on page 7 in the blades I/O guide-2009.pdf, this wording is in the box to the lower left: "4 x 10 Gb Twinax SFP+". Twinax is copper, so that might be a very subtle mention that SFP+ DA cables are available. You should get the guy who made the pdf to update it with more information regarding that though. And fix the “Qrd” typing error on page 12
erson Okay guys, see you next week to talk about Dell System Build and Update Utility 6.1