Data
Browse the data
Every (task, model) outcome in VibeSec v1. Each of the 1,000 tasks is a stable URL with the prompt, the generated app, the exploit that proves the bug, and the reference patch. Pivot the heatmap, scan the catalog, or open the trial table.
Standings
Claude Opus 4.864.9%649/1000
Claude Sonnet 4.637.3%373/1000
Kimi K2.7 Code37.2%372/1000
GLM 5.233.7%337/1000
Nemotron 3 Ultra28.9%289/1000
Mistral Medium 3.512.9%129/1000
GPT-OSS 120B11.1%111/1000
Task difficulty
265 tasks defeat every model tested; 16 are solved by all 7.
2650
1601
1542
1243
1484
825
516
167
models that solved the task
tasks × models
click a cell to open that trial
Color by
Show
Class
1,000 / 1,000 tasksSecure patchAttack still worksApp brokenRequirements changedNo response
TaskSolvedClaude Opus 4.8Claude Sonnet 4.6Mistral Medium 3.5GLM 5.2GPT-OSS 120BKimi K2.7 CodeNemotron 3 Ultra