AWS SageMaker Inference adds Nvidia MIG GPU partitioning

AWS says Amazon SageMaker Inference now supports GPU partitioning through NVIDIA Multi-Instance GPU, letting one physical accelerator be split into hardware-isolated partitions. The capability applies only to endpoints built on inference components; single-model endpoints, serverless endpoints and heterogeneous instance pools cannot be partitioned. Partition layouts are chosen when the endpoint is created, and AWS reports that carving accelerators added roughly 17 seconds to provisioning in its testing. Profiles are offered on P4d, P5, G7 and G7e instances, with no more than seven partitions per GPU. Salesforce said it helped define requirements and validated the initial implementation.