​Supercomputer leaders come together on new open-source framework

Linux has long ruled supercomputing. In the latest TOP500 supercomputer ranking, 97 percent of the world's fastest supercomputers were running Linux. There's more to running a high-performance computer (HPC) than the operating system, and open-source software is getting organized to support it.

So, on November 12th before the semi-annualSuperComputer Conference, the Linux Foundation, and a consortium on HPC leaders created the OpenHPC Collaborative Project. This project will provide a new, open source framework to support the world's most sophisticated HPC environments.

Members of the new initiative include Argonne National Laboratory, Barcelona Supercomputing Center, the Center for Research in Extreme Scale Technologies at Indiana University, Cray, Dell, Fujitsu Systems , Hewlett Packard Enterprise, Intel, the Lawrence Berkeley National Laboratory, Lenovo, Los Alamos National Laboratory, Sandia National Laboratories, SUSE, the Texas Advanced Computing Center, and many others.

HPC is used for large-scale meteorology, astronomy, engineering and nuclear physics modeling and calculations. It has unique application demands and parallel runtime requirements. Creating this software has never been easy. Supercomputing uses programming techniques, which are not used in any other kind of computing.

To make HPC software development easier, according to the Linux Foundation, "OpenHPC will provide a new, open-source framework for HPC environments. This will consist of upstream project components, tools and interconnections to enable the software stack. The community will provide an integrated and validated collection of HPC components that can be used to provide a full-featured reference HPC software stack available to developers, system administrators and users. As an open source and framework-agnostic software stack, OpenHPC will provide flexibility for multiple configurations and scalability to meet a wide variety of user needs."

Specifically, OpenHPC members will work together to:

• Create a stable environment for testing and validation: The community will benefit from a shared, continuous integration environment, which will feature a build environment and source control; bug tracking; user and developer forums; collaboration tools; and a validation environment.
• Reduce Costs: By providing an open-source framework for HPC environments, the overall expense of implementing and operating HPC installations will be reduced.
• Provide a robust and diverse open-source software stack: OpenHPC members will work together on the stability of the software stack, allowing for ongoing testing and validation across a diverse range of use cases.
• Develop a flexible framework for configuration: The OpenHPC stack will provide a group of stable and compatible software components that are continually tested for optimal performance. Developers and end users will be able to use any or all of these components depending on their performance needs, and may substitute their own preferred components to fit their own use cases.

"The use of open-source software is central to HPC, but lack of a unified community across key stakeholders--academic institutions, workload management companies, software vendors, computing leaders--has caused duplication of effort and has increased the barrier to entry," said Jim Zemlin, the Linux Foundation's executive director. "OpenHPC will provide a neutral forum to develop one open source framework that satisfies a diverse set of cluster environment use-cases."

Another drive behind this move to create a HPC open-source framework is that big business is increasingly finding uses for supercomputers. As John Russell, a HPC expert, wrote for IDC, "the collision of HPC and big data has perhaps been bigger than expected." This has resulted in the financial sector growing faster than expected. Indeed, in the last five years, supercomputing in financing has gone "into a hyper growth mode so you will see us restate our numbers for the last two years on the financial sector on the order of at least 50 percent."

So there's more of a need than ever for a standardized, open way of dealing with HPC programming. As Jerome Stoller, CTO of Big Data for Atos, said in a statement, "OpenHPC will create an ecosystem that enables collaboration between developers, system administrators and end users across a wide variety of industries. Organizations will benefit from comprehensive tested and validated HPC software stack software, a key asset to drive their own innovations forward on a stable and reliable foundation."

In short, supercomputing is no longer just about hard science. It's becoming an increasingly important part of big business. That has led to the forming of OpenHPC. This, in turn, will lead to an even greater adoption of open-source into HPC.

Source :ZDNet