ProductsNov 18, 2026, 4:09 PM EST

AWS launches SageMaker JumpStart Model Leaderboard for comparing models

AWS has added a Model Leaderboard to Amazon SageMaker JumpStart, giving SageMaker Studio users a side-by-side view of open-weight foundation models on quality, speed and cost. Quality appears as separate scores from three reasoning benchmarks — BIG-Bench Hard, GPQA Diamond and MMLU-Pro — which AWS runs itself under one evaluation setup so results are comparable across models. Speed and cost figures come from a standardized test workload sent to each model's JumpStart deployment configuration, and can be narrowed by use case profile and optimization goal. At launch the leaderboard covers more than 70 models from providers including Meta, Google, Qwen, IBM and Arcee, with more to be added over time.

Why it matters

Teams picking a model no longer have to gather benchmark numbers from scattered sources or guess at latency and running cost; every model is measured the same way, so a shortlist can be compared like for like before deployment.

Sources

Written by AIToolMaps from the source above and checked against it sentence by sentence.

Comments

No comments yet