[PCS-011] Configuring EFA and multi-NIC instances for use in AWS PCS

Brendan Bouffler
Date : August 28, 2024
Categories : AI/ML ,AWS Batch ,AWS ParallelCluster ,Amazon NICE DCV ,Elastic Fabric Adapter ,Life Sciences
Tags : ai ml ,aws ,aws pc s ,batch ,cpus ,dcv ,ec2 ,efa ,gpus ,hpc ,hpc as a service ,hpcaa s ,high performance computing ,lustre ,mpi ,nccl ,pc s ,parallel computing service ,parallel cluster ,schedulers ,storage ,autoscaling ,aws batch ,bioinformatics ,cloud computing ,elastic ,elastic fabric adapter ,hpc instances ,infiniband ,job scheduling ,scientific computing ,supercomputing ,technical computing ,tightly coupled ,virtualization ,vizualization ,techshorts

You need to do a few key things to get EFA enabled in AWS PCS clusters - and there’s an extra step or two if you’re using multi-rail EFA instances like the HPC7a or any accelerated instances from the P4 or P5 families. Matt shows us how.

Introducing AWS Parallel Computing Service, which is for customers who run high performance computing (HPC) workloads and build scientific and engineering models so they can accelerate their R&D and drive innovation.

If you have ideas for technical topics you’d like to see us cover in a future show, let us know by emailing us at ask-hpc@amazon.com.

Chapters

00:00 - Start 03:26 - Multi-NIC instances 04:38 - EFA Security Group 06:19 - Cluster Placement Group 07:39 - Launch Template 10:48 - CloudFormation template deep-dive 17:05 - What did the CFN stack create for us? 17:16 - Adding everything into PCS (final steps)