Lustre is a high-performance parallel file system commonly used in workloads requiring throughput up to hundreds of GB/s and sub-millisecond per-operation latencies, such as machine learning (ML), high performance computing (HPC), video …
Read More
When I talk with customers about their file storage, I frequently get asked “How can I determine the right throughput capacity for my file storage?” The simple answer is through monitoring the performance of your workload to determine the …
Read More
Compute queues are where all the action is on an HPC cluster. In the cloud, you get the chance to customize the queues to match the code you’re going to run on them - which is actually pretty fancy when you think about it. Matt Vaughn …
Read More
GPT, or Generative Pre-trained Transformer, is a language model that has shown remarkable progress in various vertical industries. This technology has been used to generate human-like text in fields such as finance, healthcare, legal, …
Read More
Celery is a great distributed task processing system and gets used in websites and enterprise apps everywhere. But sometimes those tasks aren’t little things that can be handled by microservices - they might need some heavy CPU or …
Read More
We always worry the most about the compute nodes on a cluster (for a good reason - it’s where all the action is), but it’s worth taking a moment to think about your head node, because that’s where you can really define …
Read More
Amazon Braket is a service that’s quite different from our other compute offerings. It’s serverless, but .. the ‘servers’ that live behind it are in cryogenic labs all around the world. This is research, done …
Read More
If you’ve ever had to stand up a rig on premises or in the cloud to run genomics pipelines, or do downstream analysis, you’ll know it’s not a small pile of packages you need to build, and connecting them all can be …
Read More