A brief History of Beowulf

High Performance Computing using cluster commodity components is generally agreed to have begun in 1993 with the Beowulf project at NASA lead by Thomas Sterling and Donald Becker. The recipe they perfected used Linux as the operating system. This choice lead to a ‘virtuous cycle’ of improvement with clusters and the cluster community contributing to Linux improving it as an operating system platform and the greater Linux community providing features to Linux that improved it as an operating system platform for cluster computing.

Near the tenth anniversary of Beowulf computing, I found and read the MIT press book “How to Build a Beowulf: A Guide to the Implementation and Application of PC Clusters”. The idea of using commodity parts to do High Performance Computing was an exciting trend and when combined with my on-going interest in parallel computing and long term interest in Linux, I committed my career to cluster based computing. As we approach the twentieth anniversary of cluster computing, I thought it would be interesting to point out that in addition to Linux based commodity clusters a second option is available for leveraging commodity hardware for HPC. That option is to use Windows NT based operating systems and the Windows ecosystem to build HPC solutions.

Windows HPC – For Real?

While some in the open source community view Windows as inappropriate for use in a cluster because it is a closed and proprietary software platform, I look at it as an alternative operating system with the volume to make it a commodity. I feel that the real spirit of Beowulf is leveraging commodity economics and accelerated progress that comes from a broad base of users and competition. I can assure you that Microsoft has felt the competitive pressure from UNIX and Linux and that this has led to many improvements in the Windows Operating system. It also led to a significant effort by Microsoft to create a full featured HPC platform based on Windows. The fruits of that effort are currently represented by a suite of products generally referred to as Windows HPC.

Windows HPC is actually implemented by combining several Windows server and client based systems into a cluster that is managed and coordinated via the Windows HPC Pack. This Windows based HPC solution is now in its third generation and provides a rich platform for doing cluster computing in a Windows environment. The latest version of Windows HPC run on dedicated on-premises hardware and can also incorporate resources in the cloud or run entirely in cloud.

Yes, VIRGINA, there is a Windows HPC solution. In future posts I will dive more deeply into the details of what the various parts of Windows HPC are and how they can be combined to meet the need for HPC in a Windows environment.

Next up: “Windows HPC – A Recipe for Success”