Amazon ECS now supports fractional GPU scheduling with Amazon EC2 G6f instances
Amazon ECS now allows fractional GPU scheduling on EC2 G6f instances, enabling smaller GPU partitions for workloads like AI inference and rendering to reduce costs.
Amazon Elastic Container Service (ECS) has introduced support for fractional GPU scheduling on Amazon EC2 G6f instances, allowing users to allocate GPU resources as small as one-eighth of an NVIDIA L4 Tensor Core GPU with 3 GB of memory. This feature targets workloads such as small-model AI inference, model experimentation, and graphics rendering that do not require a full GPU. By enabling right-sized container deployment, users can optimize infrastructure costs compared to provisioning full GPU instances. The fractional GPU allocation is specified in the container definition of an ECS task using values like GPU=0.125, GPU=0.25, or GPU=0.5.
The new capability is available on both Amazon ECS Managed Instances and Amazon ECS on EC2, providing flexibility in deployment models. ECS Managed Instances offer a fully managed experience, handling instance provisioning, scaling, patching, and lifecycle management automatically. This allows users to focus on workload execution rather than infrastructure operations. The service also includes features tailored for accelerated workloads, such as GPU metrics through Amazon CloudWatch Container Insights and automatic health monitoring to detect GPU hardware failures and replace unhealthy instances.
Fractional GPU scheduling is supported in all AWS Regions where Amazon EC2 G6f instances are available. Users can configure a capacity provider with G6f instances and specify fractional GPU values in their ECS task definitions using tools like the AWS Management Console, AWS CLI, AWS SDKs, AWS CloudFormation, or other infrastructure-as-code solutions. This integration simplifies the process of deploying GPU-accelerated workloads with precise resource allocation.
To implement fractional GPU scheduling, users must first ensure access to G6f instances in their chosen AWS Region. The configuration involves setting up a capacity provider with G6f instances and defining the required fractional GPU resources in the task definition. Additional resources, including documentation and details about G6f instances, are available on the Amazon ECS fractional GPU documentation page and the Amazon EC2 G6 instance page.