AWS adds cross-Region inference routing to Amazon SageMaker AI
AWS introduced cross-Region inference routing for Amazon SageMaker AI managed endpoints. The feature is built on a new resource called an inference router, which gives an application a single invocation path across endpoints and inference components in one or more AWS Regions and accounts. Customers register destinations for a model — such as a real-time endpoint or inference component — and invoke the router rather than a specific endpoint. The router keeps requests in its own Region while the local destination is healthy, shifts new requests elsewhere when it is not, and returns traffic gradually after recovery. Destination health is published to Amazon CloudWatch. Direct InvokeEndpoint calls still work, and the router does not create capacity, scale endpoints, or copy model artifacts.
Why it matters
Teams serving models on SageMaker AI previously had to write and maintain their own cross-Region failover logic on the critical path of every request. AWS now manages the failover thresholds and timing, and real-time requests that fail without running the model are retried once on another destination.
Sources
- PrimaryAWS Machine Learning
Written by AIToolMaps from the source above and checked against it sentence by sentence.