Cooking real clusters from the HPC Recipe Library

Brendan Bouffler
Date : April 11, 2024
Categories : AI/ML ,AWS Batch ,AWS ParallelCluster ,Amazon NICE DCV ,Elastic Fabric Adapter ,Life Sciences
Tags : ai ml ,aws ,batch ,cpus ,dcv ,ec2 ,efa ,gpus ,hpc ,high performance computing ,lustre ,mpi ,nccl ,parallel cluster ,schedulers ,storage ,autoscaling ,aws batch ,bioinformatics ,cloud computing ,cloudformation ,elastic ,elastic fabric adapter ,hpc instances ,hpc recipes ,infiniband ,job scheduling ,scientific computing ,supercomputing ,technical computing ,tightly coupled ,virtualization ,vizualization ,techshorts

Looking for a fast and easy way to deploy HPC solutions on AWS? The HPC Recipe Library has everything you need to get started.

This open-source collection of recipes lets you quickly build complex HPC infrastructure using AWS CloudFormation. You can deploy a full cluster with network infrastructure, storage systems, schedulers, and more.

The recipes are designed to work together seamlessly. For example, you can deploy a networking stack and then use its outputs as inputs for your Lustre storage stack. This modular approach makes it simple to customize your environment.

If you have ideas for technical topics you’d like to see us cover in a future show, let us know by emailing us at ask-hpc@amazon.com.