Accelerate inference with KV cache tiering on AWS

When running large language model (LLM) inference at scale on AWS, the GPU might not be the only thing that limits you. The GPU generates tokens fast, but what then contributes to performance is everything around it: memory, storage, and …

Read More

AWS Elastic Beanstalk introduces Cluster Mode

Run an application on AWS Elastic Beanstalk Cluster Mode without provisioning or operating the compute underneath it. You provide a container image or source code; Elastic Beanstalk with service-operated compute creates and operates the …

Read More

Happy 20th Birthday, Amazon EC2

On the 20th Anniversary, we recognize how AWS has continued to push the boundaries of what cloud computing can deliver, building custom silicon for general-purpose and AI workloads and expanding EC2 into new form factors and deployment …

Read More