[PCS-004] Create and use a custom AMI for AWS Parallel Computing Service

Brendan Bouffler
Date : August 28, 2024
Categories : AI/ML ,AWS Batch ,AWS ParallelCluster ,Amazon NICE DCV ,Elastic Fabric Adapter ,Life Sciences
Tags : ai ml ,aws ,aws pc s ,batch ,cpus ,dcv ,ec2 ,efa ,gpus ,hpc ,hpc as a service ,hpcaa s ,high performance computing ,lustre ,mpi ,nccl ,pc s ,parallel computing service ,parallel cluster ,schedulers ,storage ,autoscaling ,aws batch ,bioinformatics ,cloud computing ,elastic ,elastic fabric adapter ,hpc instances ,infiniband ,job scheduling ,scientific computing ,supercomputing ,technical computing ,tightly coupled ,virtualization ,vizualization ,techshorts

Custom AMIs are powerful in PCS - and also quite necessary. Matt shows us the easiest way to create one, and lets on some tricks to make it easy - and extendable later on.

Introducing AWS Parallel Computing Service, which is for customers who run high performance computing (HPC) workloads and build scientific and engineering models so they can accelerate their R&D and drive innovation.

If you have ideas for technical topics you’d like to see us cover in a future show, let us know by emailing us at ask-hpc@amazon.com.

Chapters

00:00 - Start 03:02 - Launching an instance to build on 07:57 - Reboot 10:32 - Connecting to an EFS file system 12:25 - Install SSM and CloudWatch agents 13:54 - Install PCS agent 14:56 - Install Slurm 16:22 - Installing application software via Spack 19:51 - Confirm Spack installed correctly 20:34 - Installing OpenFOAM 22:46 - Check OpenFOAM in our environment 23:56 - Build your AMI 25:17 - Add the AMI to a PCS node group