Democratize HPC Workflows

Democratize workflows to leverage High Performance Computing

HPC is often thought of as a highly specialized technology, used predominantly by a relatively small numbers of large organizations. Examples that spring to mind might include pharmaceutical companies involved in drug discovery, automotive companies performing virtual prototyping, and government organizations involved with anything from weather forecasting and epidemiology to data analysis for military or intelligence organizations. 

However, more than half of all HPC systems deployed world-wide are of relatively modest compute power, and impact a wider audience beyond the stereotypical users mentioned above. This is referred to as the “long tail” – the need for larger numbers of clusters with relatively modest computing power, perhaps in the 4- to 64-node range. 

For most organizations, the barriers to entry certainly include the complexity and cost of designing, procuring deploying, operating and giving access to HPC systems. But perhaps even more challenging is the scarcity of experts who know how to run the models that make use of HPC.

So, to support this long tail, new ways to access and run the kind of software that actually needs HPC is required.

Get In Touch

Science as a Service

Science as a Service refers to the democratization not only of HPC, but also the democratization of tools such as virtual prototyping that can take advantage of HPC. The goal is to enable analysts of all kinds to take advantage of HPC as easily and simply as, say, sending an email.

To get there, corporations need to take several steps:

  1. Identify the workflows that need HPC and whose ROI will be increased by democratized access for users.
  2. “Containerize” these workflows. According to Vanessa Sochat, of The Stanford Research Computing Center, “Containerized workloads get closer to this goal of “Science as a Service,” where it could be possible to not know a thing about servers, programming, or containers, but be a really impeccable scientist that can write grants, get funded, collect data, and then analyze it with optimally developed pipelines delivered via containers.” (Full article here). A guiding principle for the container design is to ensure that the workflow is totally portable and deployable on the selected public and / or private cloud HPC infrastructures.
  3. Specify and deploy a self-service HPC infrastructure.
  4. Organize governance and dissemination of the containerized workflows powered by the self-service HPC infrastructure.

The point is that to leverage HPC, users traditionally needed to be expert not only in HPC architecture and operating systems, but also in the specific modeling and simulation tools to be run on a cluster. Even in large organizations that can afford experts with the required skillsets, the need for expertise is a barrier for the vast majority of potential HPC users, reducing the ROI and potentially impacting the organization’s competitiveness. In smaller organizations, it can mean that HPC is simply not an option.

Leveraging the power of HPC using EASA

Using EASA, Pfizer have built a number of web applications that democratize not only complex modeling and simulation software, but also access to the required HPC infrastructure.

Read Case Study

Democratize access to HPC by creating custom web apps in a no-code environment

EASA is one of several technologies that can help overcome this barrier, enabling companies to ramp up usage of HPC, and thus increase the return on investment. Using the EASA platform, companies rapidly build custom web apps that drive even the most “expert-only” modeling tools, automatically submitting jobs either directly to a cluster, or to an existing HPC queuing system. These web apps enable users to define the problem to be solved in terms that they understand, enabling non-experts to safely utilize sophisticated modeling tools and the HPC clusters on which they run. 

Because EASA “democratizes” access to the kind of modeling tools that actually need HPC, a far greater number of users can take advantage – which in turn leads to greater demand for HPC resources. 

HPC and Machine Learning

The rapid advances in Machine Learning represent a promising new approach to solving challenging problems in engineering, science, and business. In particular, data from instruments and products has never been so accessible, so the science of physics-based modelling can now be improved with Deep Learning and other Artificial Intelligence techniques. 

While most trained ML models are generally highly performant and have no need of HPC, the initial training of a model often requires significant compute-resources. In some cases, models may need to be re-trained as new data sets become available, and this can become a bottle-neck if the only people that can do it are highly experienced data scientists with knowledge of HPC.

Again, EASA can be used to build – at very low cost – customized apps that enable the non-expert user to execute ML model training, while automating access to HPC resources in the process. 

Free Project Consultation

Request a free project consultation with EASA so we can discuss your requirements and, if desired, provide you with a demonstration of the software.

Enquire Now

Solutions

Common Uses Cases

Resources

Company

Facebook Youtube Linkedin