Overview of services built for HPC customers @ AWS

Brendan Bouffler
Date : March 14, 2024
Categories : AI/ML ,AWS Batch ,AWS ParallelCluster ,Amazon NICE DCV ,Elastic Fabric Adapter ,Life Sciences
Tags : ai ml ,aws ,batch ,cpus ,dcv ,ec2 ,efa ,gpus ,hpc ,high performance computing ,lustre ,mpi ,nccl ,parallel cluster ,schedulers ,storage ,autoscaling ,aws batch ,bioinformatics ,cloud computing ,elastic ,elastic fabric adapter ,hpc instances ,infiniband ,job scheduling ,scientific computing ,supercomputing ,technical computing ,tightly coupled ,virtualization ,vizualization ,techshorts

Feel limited by the computing resources available to you locally? Are you looking for ways to supercharge your research and engineering projects?

This tech short video introduces powerful cloud-based tools that can help you achieve the near impossible.

Learn how AWS services can help remove the barriers holding you back.

We’ll explain how these tools allow you to spin up resources right away, collaborate seamlessly, and run workloads at scales previously unimaginable.

Whether you’re a researcher, engineer, or admin, this video demystifies HPC in the cloud.

If you have ideas for technical topics you’d like to see us cover in a future show, let us know by emailing us at ask-hpc@amazon.com.

Chapters:

00:00 - How we think about HPC 00:50 - Compute, Storage, Networking building blocks 01:05 - Compute (HPC and GPU instances) 01:42 - Networking for HPC (EFA) 02:48 - Storage (Lustre, OpenZFS) 03:40 - AWS ParallelCluster (HPC clusters) 04:20 - AWS Batch (container batch workloads) 05:14 - Workflow tool integrations 05:30 - Research and Engineering Studio (RES) 07:03 - HPC recipes library