With AWS ParallelCluster 3.6, you can directly specify Slurm settings in the cluster config file – improving reproducibility and another step towards self-documentation for your HPC infrastructure. Read the Post on the AWS Blog Channel
Read More
Scientific software is almost defined by it’s complexity. But just because it was hard to write doesn’t mean it should be hard to build and even harder to run well. EESSI is a project that aims to change that by making …
Read More
Building scientific or engineering code stacks from source used to suck in the time before Spack. Todd, Greg and the community built an amazing resource that houses all the build recipes and knows all the dependencies SO YOU DON’T …
Read More
Building ParallelClusters that span multiple Availability Zones in a region take some careful planning with respect to storage and networking, but opens up options for more capacity - which is often the driver behind this design. Matt …
Read More
AWS ParallelCluster 3.6.0 can now detect GPU failures in HPC and AI/ML tasks. Health checks run at the start of Slurm jobs and if they fail, the job is requeued on another instance. This can increase reliability and prevent wasted spend. …
Read More
Compute queues are where all the action is on an HPC cluster. In the cloud, you get the chance to customize the queues to match the code you’re going to run on them - which is actually pretty fancy when you think about it. Matt Vaughn …
Read More
Celery is a great distributed task processing system and gets used in websites and enterprise apps everywhere. But sometimes those tasks aren’t little things that can be handled by microservices - they might need some heavy CPU or …
Read More
We always worry the most about the compute nodes on a cluster (for a good reason - it’s where all the action is), but it’s worth taking a moment to think about your head node, because that’s where you can really define …
Read More