Moving your /home off the cluster in ParallelCluster

Brendan Bouffler
Date : April 4, 2024
Categories : AI/ML ,AWS Batch ,AWS ParallelCluster ,Amazon NICE DCV ,Elastic Fabric Adapter ,Life Sciences
Tags : home ,ai ml ,aws ,batch ,cpus ,dcv ,ec2 ,efa ,gpus ,hpc ,high performance computing ,lustre ,mpi ,nccl ,parallel cluster ,schedulers ,storage ,autoscaling ,aws batch ,bioinformatics ,cloud computing ,elastic ,elastic fabric adapter ,hpc instances ,infiniband ,job scheduling ,scientific computing ,storage ,supercomputing ,technical computing ,tightly coupled ,virtualization ,vizualization ,techshorts

In this video, learn how decoupling key filesystems from your head node with Amazon EFS can prevent meltdowns from pathological jobs.

Plus, provision persistent shared storage for /home directories across cluster lifecycles. Join us as we dive into the simple configuration changes that unlock a world of potential.

If you have ideas for technical topics you’d like to see us cover in a future show, let us know by emailing us at ask-hpc@amazon.com.