OpenAI publishes perplexity data and proposes shared safety limits
OpenAI Chief Scientist Jakub Pachocki published a research and safety post arguing that perplexity, also called cross-entropy loss, should become a shared metric for comparing model developers and setting safety standards. As a first step, OpenAI released internal perplexity measurements for its models starting with GPT-3.5. Pachocki points to capability changes at lower log-perplexity values, including work in math and cybersecurity, and expects more in the 0.68 to 0.7 range. The post proposes industry norms for evaluation and red-teaming, plus common development limits tied to perplexity until monitoring reaches a concrete milestone. The essay and measurements are public; no new model or product is announced.
Sources
- PrimaryOpenAI
Written by AIToolMaps from the source above and checked against it sentence by sentence.