Showing items from Schedulers

Sending Celery tasks to AWS Batch for heavy lifting

Celery is a great distributed task processing system and gets used in websites and enterprise apps everywhere. But sometimes those tasks aren’t little things that can be handled by microservices - they might need some heavy CPU or …

Read More

Six tips for cluster head node design

We always worry the most about the compute nodes on a cluster (for a good reason - it’s where all the action is), but it’s worth taking a moment to think about your head node, because that’s where you can really define …

Read More

A fast walk through Amazon Omics

If you’ve ever had to stand up a rig on premises or in the cloud to run genomics pipelines, or do downstream analysis, you’ll know it’s not a small pile of packages you need to build, and connecting them all can be …

Read More