Researchers release MMDM benchmark for AI decision models

A new arXiv paper introduces MMDM, a benchmark for measuring decision models, the systems that output structured probabilistic choices in a single forward pass instead of writing out reasoning or text.

The authors evaluated 28 decision models and report that many do well on common knowledge while struggling with interdependent options, constrained decisions and multi-select questions. Perplexity Decider-V1.2 scored 78.75% hard single-select accuracy on world knowledge, 53.95% on option logic and 47.95% on constraint planning, and the best hard multi-select accuracy across all models was 36.81%.