Collinear AI releases CWE-bench v2 with 100 security tasks

Collinear AI has published CWE-bench v2, a public benchmark of 100 security tasks covering 91 different weakness classes, announced on the Hugging Face blog. Each task gives an agent a checkout of a real open-source repository and a short instruction to find and fix a vulnerability without breaking existing functionality, and all scoring is programmatic, so results are deterministic and reproducible at no extra cost. A held-out set is available by request. The public set, grader and harness are open, with every task shipped as a Dockerized Harbor environment and tagged as an RL environment that can also reward training defensive agents. On the project's leaderboards, Gemini 4 Argon leads with 67.8% pass@1 and 81.0% pass@4, just ahead of Claude Opus 5.5, and 95 tasks are solved by at least one of the 11 evaluated models.