Mark Fernandes
Dell's Dr. Mark Fernadez

In the last installment of this blog, we introduced the Hierarchical Storage Management System or HSM, and discussed some of its high level characteristics. In this segment, we will cover additional details of an HSM, its roles and responsibilities.

In practice, suppose that adequate high-speed HPC storage were available for your compute jobs when you needed it, as well as available to others when they needed it. With proper planning, proper expectations and an HSM, this can be done. Just as your compute jobs are moved in and out of valuable HPC compute resources based upon some policies and a software package commonly called a job-scheduler, and just as those valuable HPC compute resources are shared with others, so can the high-speed HPC storage in a tiered storage environment. With an HSM, the benefits of high-speed (high-cost) HPC storage can be linked to more affordable, often larger capacity storage. This tiering of storage is one of the key features of an HSM.

An HSM, as mentioned previously, continuously monitors and automatically moves files and data between different storage tiers; and allows files and data to always appears to be online to end users and applications regardless of the actual storage location. As the need for the files or data arises, the HSM moves the files or data from their low cost, high capacity location to the high-speed HPC storage. Once the need is fulfilled, the files and data can be moved by the HSM from the high-speed HPC storage to a more appropriate (less costly and greater capacity) storage subsystem. Just like the job-scheduler makes room for the next job on the compute resources, the HSM makes room for the next job on the high-speed storage.

In the HSM world, the terms used to describe this movement between storage tiers are “migration” and “recall.” By migrating files and data to more affordable, often larger capacity storage until needed, an adequate high-speed HPC storage capacity can be maintained for the jobs currently running. When files and data are needed for HPC computations, they are recalled to their original locations on the high-speed storage. Hence, the overall costs and the total storage capacity can be maximized within a given budget.

In the next part of this blog, we will distinguish an HSM from other storage software solutions. If you have comments or can contribute additional information, please feel free to do so.

Thanks. --MRF