Clusters have become one of the most popular architectures for High Performance Computing (HPC) today. Along with the increasing popularity in HPC architectures, the HPC storage market has witnessed an explosion in data growth, presenting an increasing concern for data management. Enterprise IT departments are searching for solutions to address the data growth and regulatory/compliance requirements, while limiting the effect on IT budgets and resources. Data archiving and management are key emergent areas to address the rising data storage needs effectively.
The backbone of a Hierarchical Storage Management (HSM) solution is the software. The HSM system monitors the way data is used, then through defined storage policies, moves less frequently used or lower priority data safely to slower tiers, reclaiming storage space on faster tiers. Very detailed logic is required to catalog the data and watch for frequently used files that should reside on the upper tier. The software is also responsible for managing the requests and ensuring those requests happen in a timely manner.
There are key features needed in an HSM solution: Policy Based Migration and Transparent Access.
Rules definable by thresholds, owners, age, size, filters, folders or other search criteria allow added flexibility. Data can move from its existing tier space to a different tier space, or copy to another tier, thereby enabling data protection within the same solution.
End users maintain their access as before the archiving or HSM occurred. Also, the user can search and recall the information from the archive or stub file in the originating tier in the HSM scenario, minimizing the help needed from administrators.
Enter the Dell HPC NSS Tiered Storage Solution (HTSS)!!!
The Dell HPC Tiered Storage Solution integrates CommVault Simpana 9 Data & Information Management Software with the Dell NFS Storage Solution (NSS) and PowerVault ML6000 Tape Libraries to provide a multi-tiered Hierarchical Storage Management system. The goal is to provide a data storage technique that offers Hierarchical Storage Management (HSM) and archive solutions that automatically move data between storage tiers through the use of defined storage policies.
Customers are seeking effective solutions to:
HPC Customers typically have three kinds of needs for an HPC storage solution.
Scratch Space (tier1): high throughput and scalable cluster working space. This solution is provided by the Dell | Terascala HPC Storage Solution (DT-HSS), which offers a high throughput scale-out storage appliance based on the Lustre file system and Dell PowerVault storage arrays.
Primary Storage (tier2): reliable, cost effective and good performance storage for user data. This solution is provided by the Dell NFS Storage Solution (NSS) that uses the NFS file system on top of the Red Hat Scalable File System (XFS) with Dell PowerVault storage. It provides an easy to manage, reliable, and cost-effective solution for unstructured data.
Long-Term Storage (tier3): low cost, high capacity storage for long-term retention. Long-term storage is disk based, tape based, or a combination of both.
The Dell HPC Tiered Storage Solution enables Dell customers to manage the archival and movement of their data between these tiers.
More specifically, the HTSS leverages sophisticated, policy-based data management automation processes to move data from tier to tier, leaving behind a small stub file containing the file metadata and moving the data blocks to a lower tier (commonly referred to as HSM). Additionally, you can use it for archiving purposes where no file stub is left behind and a new method of accessing the data is required. Typically, frequently used data can be moved to a faster tier, while infrequently used data can be moved to slower media at lower tiers. Correspondingly, as files are used more often, they can reside on the first or fastest tier and then moved down to the slowest or lowest tier as they fall out of use, but must be retained to meet regulatory or compliance requirements.
The management of all this data movement is centralized to CommVault Simpana 9 Data Management Software in a Hierarchical Storage Management (HSM) implementation. The solution is delivered as an all-inclusive storage data management solution and is available with deployment services and full hardware and software support from Dell and CommVault.
For more detailed information on the HTSS solution, including detailed cookbook configuration, performance and tuning details, pick up your copy of the Dell HTSS Whitepaper today!