Dell Community

Blog Group Posts
Application Performance Monitoring Blog Foglight APM 105
Blueprint for HPC - Blog Blueprint for High Performance Computing 0
CommAutoTestGroup - Blog CommAutoTestGroup 1
Custom Solutions Engineering Blog Custom Solutions Engineering 5
Data Security Data Security 8
Dell Big Data - Blog Dell Big Data 68
Dell Cloud Blog Cloud 42
Dell Cloud OpenStack Solutions - Blog Dell Cloud OpenStack Solutions 0
Dell Lifecycle Controller Integration for SCVMM - Blog Dell Lifecycle Controller Integration for SCVMM 0
Dell Premier - Blog Dell Premier 3
Dell TechCenter TechCenter 1,854
Desktop Authority Desktop Authority 25
Featured Content - Blog Featured Content 0
Foglight for Databases Foglight for Databases 35
Foglight for Virtualization and Storage Management Virtualization Infrastructure Management 256
General HPC High Performance Computing 223
High Performance Computing - Blog High Performance Computing 35
Hotfixes vWorkspace 57
HPC Community Blogs High Performance Computing 27
HPC GPU Computing High Performance Computing 18
HPC Power and Cooling High Performance Computing 4
HPC Storage and File Systems High Performance Computing 21
Information Management Welcome to the Dell Software Information Management blog! Our top experts discuss big data, predictive analytics, database management, data replication, and more. Information Management 229
KACE Blog KACE 143
Life Sciences High Performance Computing 5
OMIMSSC - Blogs OMIMSSC 0
On Demand Services Dell On-Demand 3
Open Networking: The Whale that swallowed SDN TechCenter 0
Product Releases vWorkspace 13
Security - Blog Security 3
SharePoint for All SharePoint for All 388
Statistica Statistica 24
Systems Developed by and for Developers Dell Big Data 1
TechCenter News TechCenter Extras 47
The NFV Cloud Community Blog The NFV Cloud Community 0
Thought Leadership Service Provider Solutions 0
vWorkspace - Blog vWorkspace 510
Windows 10 IoT Enterprise (WIE10) - Blog Wyse Thin Clients running Windows 10 IoT Enterprise Windows 10 IoT Enterprise (WIE10) 3
Latest Blog Posts
  • Life Sciences

    Scaling behavior of short read sequence aligners

    This blog explores the scaling behaviors of the most popular three short read sequence aligners; BWA-mem, Bowtie and Bowtie2. Aligning process is the beginning of all Next Generation Sequencing (NGS) data analysis and typically the most time-consuming step in any NGS data analysis pipeline.

    It is clear that using more cores for the aligning process will help to speed up the process, but as you already know parallelization comes with cost. Higher complexity, perhaps order of magnitude, due to multiple instruction streams and data flowing between the instruction streams can easily overshadow the speed gain by parallelizing the process. Hence, it is not wise to use the entire cores in a compute node to speed up blindly especially when the overall throughput is more important. Identifying a sweet spot for the optimal number of cores for each aligning process and maximizing the overall throughput at the same time is not an easy task in a complex workflow.

    Table 1 Server configuration and software

    Component

    Detail

    Server

    PowerEdge FC430 in FX2 chassis - 2 Sockets

    Processor

    Intel® Xeon® Dual E5-2695 v3 - 14 cores, total 28 physical cores

    Memory

    128GB - 8x 16GB RDIMM, 2133 MT/s, Dual Rank, x4 Data Width

    Storage

    480TB IEEL (Lustre)

    Interconnect

    InfiniBand FDR

    OS

    Red Hat Enterprise 6.6

    Cluster Management tool

    Bright Cluster Manager 7.1

    Short Sequence Aligner

    BWA 0.7.2-r1039, Bowtie 1.1.2, Bowtie 2.2.6

     

    Table 1 shows the summary of the system for the test here. Hyperthreading was not enabled for the test although it helps to improve the overall performance.

    In Figure 1, BWA running times were measured for each different sizes of paired end read sequencing data. The size of sequence read data is represented as million fragments (MF) here. For example, 2MF means that the sequence data consist of two fastq files containing two million sequence reads in each file. One read is in a fastq file, and the corresponding paired read is in the other fastq file. A sweet spot for BWA-mem is from 4 to 16 cores roughly depending on the sequence read file size. Typically, the sequence read size is larger than 10 million and hence, 2MF and 10MF results are not realistic. However, larger sequence read data follow the behavior of the smaller input sizes as well. In the speedup chart, blue solid line labeled as 'ideal' represents the theoretical speedup we could obtain by increasing the number of cores.

      

    Bowtie results in Figure 2 shows that the similar sweet spot comparing to BWA-mem results. However, the running time is slightly faster than BWA-mem’s. It is notable that Bowtie and Bowtie2 are sensitive to the read lengths while BWA-mem shows more consistent scaling behavior regardless of the sequence read lengths.

     

    Although Bowtie2 is even faster than Bowtie, it is more sensitive to the length of the sequence reads as shown in Figure 3. Actually, the total number of nucleotides could be a better matrix to estimate the running time for a given sequence read data.

    In practice, there are more factors to consider to maximize the overall throughput. One of critical factors is the bandwidth of existing storages in order to utilize the sweet spot instead of using the entire cores in a compute node. For example, if we decided to use 14 cores for each aligning process instead of using 28 cores, this will double up the number of samples processed simultaneously. However, the twice number of processes will fill up the limited storage bandwidth, and overly used storages will slow down significantly the entire processes running simultaneously.

    Also, these aligning processes are not used alone typically in a pipeline. It is frequently tied up with a file conversion and a sorting process since subsequent analysis tools requiring these alignment results sorted in either chromosome coordinates or the name of the reads. The most popular approach is to use ‘pipe and redirection’ to save time to write multiple output files. However, this practice makes the optimization harder since it generally requires more computational resources. More detailed optimization for NGS pipelines in this aspect will be discussed in the next blogs.

  • Dell TechCenter

    Dell PowerEdge servers with Intel’s Xeon E5-2600/4600 deliver top scores in HEPSPEC 06 for Scientific Linux 6 and Scientific Linux 7!

    Dell PowerEdge R730 and FC830 servers available with the Intel Xeon Processor E5-2600/4600 v4 product family deliver top scores in HEPSPEC06.

     Benchmarking of the PowerEdge R730 and FC830 servers confirms that it delivers the full potential of the new architecture by attaining world records in HEPSPEC06.

    Performance increases of 24 to 33% can be achieved in HEPSPEC06 when going from E5-2600/4600v3 to E5-2600/4600v4 processors.

    HEPSPEC 06 benchmark info and results can be found at the following website. http://w3.hepix.org/benchmarks/doku.php

      

     

     

    Claim based on best published four-processor, two-processor HEPSpec 06 Standard Application Benchmark result using the Linux* operating system published at http://w3.hepix.org/benchmarks/doku.php?id=bench:results_sl7_x86_64_gcc_48x for Scientific Linux 7 and http://w3.hepix.org/benchmarks/doku.php?id=bench:results_sl6_x86_64_gcc_445 for Scientific Linux 6. Configuration: Dell PowerEdge* FC830 platform with four Intel® Xeon® processor E5-4669 v4 (22 cores, 44 threads, for a system total of 88 cores, 176 threads),  Scientific Linux 7.2, Scientific Linux 6.7. Dell PowerEdge* R730 platform with two Intel® Xeon® processor E5-2699 v4 (22 cores, 44 threads for a system total of 44 Cores and 88 threads),  Scientific Linux 7.2, Scientific Linux 6.7.

  • Dell TechCenter

    Dell OpenManage Plug-In for Nagios XI is Available Now!!

    We’re excited to bring the latest release of Dell OpenManage Plug-In for Nagios XI to you. With the launch of this plug-in, we are now extending our Nagios-Core plugin for Nagios XI users as well with some great new feature too. 

    Dell OpenManage Plug-in version 1.0 for Nagios XI provides capabilities to monitor 12th and later generations of Dell PowerEdge servers through an agent-free method using Integrated Dell Remote Access Controller (iDRAC) with Lifecycle Controller, Datacenter Scalable Solutions, Dell chassis and Dell Storage devices in the Nagios XI console. This plug-in provides comprehensive hardware-level visibility including overall and component-level health monitoring of Dell PowerEdge servers through SNMP and WS-MAN protocols, Dell chassis through WS-MAN protocol and Dell storage through SNMP protocol. This plug-in provides basic information about the Dell devices and its components and also monitors the events that are generated from the Dell devices. This plug-in also supports one-to-one web console launch for iDRAC, Chassis, and storage devices to perform further troubleshooting, configuration, and management activities.

    What you can do with Dell OpenManage Plug-in for Nagios XI?

     

    Feature

    Function

    Benefit

    Dell Configuration Wizard

    Automated and Guided step-by-step web configuration wizard for Discovery and Monitoring of Dell device including device component health (e.g. Physical disk, Virtual disk, Fan, Battery, NIC, and Intrusion etc.)

    GUI based wizard enables quick monitoring configuration from a single interface. Choose from basic as well detailed monitoring without getting into the nitty-gritty of command-line scripting.

    Event monitoring

    Monitor SNMP traps from the supported Dell devices.

    Quickly gather high-level information such as overall health as well component-level health of your Dell devices for immediate troubleshooting.

    Deep-Level Hardware Inventory

    Comprehensive device information including deep-level component-level details.

    Keep a pulse on your mission-critical data center infrastructure assets and periodically monitor them

    Link and Launch Device Consoles

    Launch One-to-One Integrated Dell Remote Access Controller (iDRAC) console for Dell PowerEdge Servers as well as One-to-Many consoles for Dell Storage devices directly from Nagios XI.

    Enables further troubleshooting and one-to-one configuration, update, or management of Dell devices.

    Warranty Information

    Monitors and displays the warranty information for the supported Dell devices.

    Proactively monitor the complete hardware warranty information of the Dell devices.

    For more information, download links, and product documentation please visit the Dell OpenManage Plug-In for Nagios XI wiki page. We encourage you to continue this conversation in the OpenManage Connections for 3rd Party Console Integration Forum if you have any comments or other feedback.

  • Dell TechCenter

    Dell 13G PowerEdge servers now with Intel’s latest Xeon E5-2600 v4 processors, deliver on our relentless pursuit to produce more work with less energy.

    By Bruce Wagner

    Date: July 15, 2016 at 1:45 PM Central Time

    Dell Tech Center Blog (EXTERNAL)

    Performance benchmarking of this new PowerEdge R630 rack server, configured just as many data center customers prefer, confirms Dell remains committed to its goal of providing highest possible energy efficiency solutions.

    • 1U rack server configuration of highest order acceptance. 13G+ configuration included: 2 x Xeon E5-2620v4 CPU, 64GB DDR4 memory, 2 x 300GB 10K SAS RAID1 storage, 4 x 1GbaseT ETN LOM ports, redundant PSU.

    • Costing analysis based upon:
    1. US EIA 2016 average retail price of electricity for commercial sector in ($ per KWh) = $0.10.  http://www.eia.gov/electricity/monthly/epm_table_grapher.cfm?t=epmt_5_6_a
    2. The fully burdened cost of a watt (annualized) to operate a data center computer server = $2.69  http://perspectives.mvdirona.com/2008/12/06/AnnualFullyBurdenedCostOfPower.aspx

  • Information Management

    The Information Management Community Has Moved! Come See Our New Home.

    It's the moment we've all been waiting for!  We've launched our brand new Systems and Information Management (SIM) Community!
    If you're looking for new content around Database Management, Big Data, and Product News for SharePlex, Toad, Stat and more, head over to the new site!  We know you may have some questions.  How did we get here? What we are doing, when we're doing it, and why? 
    Read about all of the new features with in the SIM Community and learn how our community seeks to give you the information you need. Please check out our new blog.

    We promise to give you a more focused user experience in our new community where you can get the information you want through an easier navigation. If you have suggestions on how we can make our new home better, post your thoughts on our Feedback Forum.

    And if the new SIM Community sounds intimidating, don't worry, here's "Everything you need to know to get started in the SIM community!"

    About Ryan McKinney

    Ryan McKinney has been a Social Media and Communities Advisor for Dell Software since 2014.

    View all posts by Ryan McKinney | Twitter

  • KACE Blog

    Why We're Moving to a New Community. Hint: It's About Our Customers

    Today, the new SIM Community is live! Please join us on our new home for all Systems and Information Management content. Read about all of the new features with in the SIM Community and learn how our community seeks to give you the information you need. Please check out our new blog.

    We promise to give you a more focused user experience in our new community where you can get the information you want through an easier navigation. If you have suggestions on how we can make our new home better, post your thoughts on our Feedback Forum.

  • vWorkspace - Blog

    It’s Time to Innovate Together!

    As of today, the new SIM Community is live! Please join us on our new home for all Systems and Information Mgmt. content. You can read about all of the new features with in the SIM Community and learn how our community seeks to give you the information you need. Please feel free to check out our new blog as well.

    We promise to give you a more focused user experience in our new community where you can get the information you want through an easier navigation. If you have suggestions on how we can make our new home better, I invite you to post your thoughts on our Feedback Forum.

  • Desktop Authority

    Desktop Authority: It’s Time to Innovate Together!

    As of today, the new SIM Community is live! Please join us on our new home for all Systems and Information Mgmt. content. You can read about all of the new features with in the SIM Community and learn how our community seeks to give you the information you need. Please feel free to check out our new blog as well.

    We promise to give you a more focused user experience in our new community where you can get the information you want through an easier navigation. If you have suggestions on how we can make our new home better, I invite you to post your thoughts on our Feedback Forum.

  • Application Performance Monitoring Blog

    Performance Monitoring: It’s Time to Innovate Together!

    As of today, the new SIM Community is live! Please join us on our new home for all Systems and Information Mgmt. content. You can read about all of the new features with in the SIM Community and learn how our community seeks to give you the information you need. Please feel free to check out our new blog as well.

    We promise to give you a more focused user experience in our new community where you can get the information you want through an easier navigation. If you have suggestions on how we can make our new home better, I invite you to post your thoughts on our Feedback Forum.

  • Foglight for Databases

    Foglight for Databases: It’s Time to Innovate Together!

    As of today, the new SIM Community is live! Please join us on our new home for all Systems and Information Mgmt. content. You can read about all of the new features with in the SIM Community and learn how our community seeks to give you the information you need. Please feel free to check out our new blog as well.

    We promise to give you a more focused user experience in our new community where you can get the information you want through an easier navigation. If you have suggestions on how we can make our new home better, I invite you to post your thoughts on our Feedback Forum.