[PCS-015] Bring your own login nodes for AWS Parallel Computing Service

Brendan Bouffler
Date : August 28, 2024
Categories : AI/ML ,AWS Batch ,AWS ParallelCluster ,Amazon NICE DCV ,Elastic Fabric Adapter ,Life Sciences
Tags : ai ml ,aws ,aws pc s ,batch ,cpus ,dcv ,ec2 ,efa ,gpus ,hpc ,hpc as a service ,hpcaa s ,high performance computing ,lustre ,mpi ,nccl ,pc s ,parallel computing service ,parallel cluster ,schedulers ,storage ,autoscaling ,aws batch ,bioinformatics ,cloud computing ,elastic ,elastic fabric adapter ,hpc instances ,infiniband ,job scheduling ,login node ,scientific computing ,supercomputing ,technical computing ,tightly coupled ,virtualization ,visualization ,vizualization ,techshorts

You can build login nodes that connect to AWS PCS clusters - even if they’re not part of the cluster! This is super helpful if you want to give each one of your users access to their own personal login node - to keep their CPU and I/O usage from impacting each other.

You can also use this to cook visualization nodes with amazing remote viz tech like DCV.

Introducing AWS Parallel Computing Service, which is for customers who run high performance computing (HPC) workloads and build scientific and engineering models so they can accelerate their R&D and drive innovation.

If you have ideas for technical topics you’d like to see us cover in a future show, let us know by emailing us at ask-hpc@amazon.com.

Chapters

00:00 - Start 01:23 - What’s involved in creating a login node? 03:10 - Hands on tutorial 03:28 - Metadata gathering 04:15 - Supported distros 05:41 - Security groups 07:39 - Why SSM? 08:11 - Connecting to the instance 09:26 - Installing Slurm 11:07 - Configure sackd