Security code review
Results are only compared within a matching group, so a model tested one way is never ranked against a model tested another way.
Find security vulnerabilities in real application code and fix them without breaking legitimate behavior
Ranking groups use separate scales. Compare configurations only inside the same group; EvalRank never flattens unlike systems into one leaderboard.
Cross-benchmark consensus
One ranking of models across this topic's admitted benchmarks, weighted the same way decisions are. Each score is the model's share of the best result.
| Rank | Model | Share of best | Benchmarks |
|---|---|---|---|
| 1 | Kimi K3.1 Pro | 100.0% | 2 |
| 2 | MiMo-V2.6-Pro | 87.5% | 2 |
| 3 | GLM-5.3-Flash | 72.6% | 2 |
| 4 | Step 5 Preview | 64.3% | 2 |
| 5 | Beam 1.0 | 49.2% | 2 |
| 6 | Qwen3.8 2.4T A95B | 36.0% | 2 |
| 7 | MiniMax-M3 | 35.0% | 2 |
| 8 | Nemotron 3 Ultra 550B A55B | 23.6% | 2 |
Agent system · Agentic
Exploratory - not a ranked winner yetrg-security-code-review-agent-system-agentic-agent-system-v1Explorer evidence
CWE-Bench-AA (Artificial Analysis)
Sources differ; shown separately, never averaged. Observed ; fresh until .
| Rank | Configuration | Capability | Uncertainty | Families |
|---|---|---|---|---|
| 1 | DeepSeek V4.1 Flash 1017 (Max) cwe-bench-aa:f151fd0b-2139-40cf-a3ec-995e8cedc592config_b4568a44ba7d21a439edb6a56d1dd5bcae41e3cc0d1e7c6197f12cca591d4556 | 100% | Unknown | 1 |
| 2 | Grok 4.7 (Xhigh) cwe-bench-aa:59627802-1546-422c-9a65-65b61c022d36config_31b6c76643164be43b610600471ddb316446ebf1e6a9ad852853f1e470fb4199 | 98% | Unknown | 1 |
| 3 | GPT-6 Sol (Daybreak Blue, max) cwe-bench-aa:93e12518-6e04-4713-b52d-2c56dac49c63config_247334b28fd1aed37fa74acbd03ab41a7719f448751b986a3c5ca641902927bf | 90% | Unknown | 1 |
| 4 | GPT-6 Astra (Max) cwe-bench-aa:2f339a97-9a0d-499a-9cb5-e0db665bfa25config_0ff6c2e86d5fc3f8b315d1423793e587e2f52c6c7ae7e79ab9b3013eab9e95d2 | 86% | Unknown | 1 |
| 5 | MiMo-V2.6-Pro cwe-bench-aa:24d8fdfe-6241-424e-b7b7-1b6efb06e4fbconfig_78ea99edad6aab69cc66343588172e0229061ad766e52e4e070f86eeaa04fde9 | 86% | Unknown | 1 |
| 6 | Kimi K3.1 Pro cwe-bench-aa:665c67ca-09eb-48f0-a7a3-0096e1d5de51config_9cb2282d6a43fd40030819601e648892d6c1cd57e88bf0b09c92ed6c170870a2 | 80% | Unknown | 1 |
| 7 | Claude Opus 5.5 (Max, Default Fallback) cwe-bench-aa:2f3c4dc9-a450-4303-8697-0237257cf08fconfig_75aaaf8b916b7a66d01b00ce5506718dfaed135a097bd26331afc83e9ecf4ac1 | 75% | Unknown | 1 |
| 8 | Mistral Large 4 (High) cwe-bench-aa:ea5019ae-b0b7-49a5-80b3-0e584807af0fconfig_a03f5fba6bc3ec3837542b12cce927f15c1b58df7c0e7fab8ce2ea0eecff3f56 | 75% | Unknown | 1 |
| 9 | GPT-6 Luna (Max) cwe-bench-aa:6fb13851-40ce-4bda-ad3f-6b38e0c5daa7config_93cace1581371fb001fba9498b62acaf028700c5d458e2ef6afcc240ee4dd573 | 71% | Unknown | 1 |
| 10 | GLM-5.3 (Max) cwe-bench-aa:cd684ea4-b475-4269-b001-d469d06d8a7aconfig_3325cc082d7df8068a9cc64cf63782c025776594aa02f9958e0fd96e482ceb18 | 69% | Unknown | 1 |
| 11 | GLM-5.3-Flash cwe-bench-aa:19496b81-9f41-4214-a77a-1df803b3c5aeconfig_9f130b98f6a0318d0dcfd75f2dbad7ebf3128e51dca686e27609f3e179f9c8bb | 69% | Unknown | 1 |
| 12 | Gemini 3.8 Flash (High) cwe-bench-aa:8a999846-4c1d-4ce7-a8b7-1310a7166fd7config_354b3e670d3aa257f24e6d3e5c7badd5feb44c68dee0982822a4e0922dfad597 | 61% | Unknown | 1 |
| 13 | Step 5 Preview cwe-bench-aa:094ea3e3-ce41-419e-8bd1-723389663158config_67de31038e4fe36bc933bc68fc501555cd2288c2083f73088ac2ce3a04a5b7a2 | 61% | Unknown | 1 |
| 14 | Claude Fable 5.1 (Max, Default Fallback) cwe-bench-aa:3e87c73e-a257-495e-9730-367a66229811config_d8d4e1e7689af7c42f5688d542d5fb95f1a6b24903a29c9d49326ae537ee7b60 | 57% | Unknown | 1 |
| 15 | Beam 1.0 cwe-bench-aa:69468cce-7b5c-414a-b498-41df4c34df85config_b9982c75359bd843bfbd4ddd068638eb6f66a4c60c3d2d7d085e2d26230e50c1 | 53% | Unknown | 1 |
| 16 | MiniMax-M3 cwe-bench-aa:277f939a-985b-4b37-859d-b3eabc7c0b26config_907105a30b99df99fd3e6f83c042c3dbe0ca75fcc5bc8f41d816424fb62b8ecc | 35% | Unknown | 1 |
| 17 | Qwen3.8 2.4T A95B cwe-bench-aa:8df710d3-9dae-4498-9b4e-9818238e6f31config_5088d33050ecbde1e905be1ccaca5d512398e5eb11772e6788e5090614251df8 | 4% | Unknown | 1 |
| 18 | Nemotron 3 Ultra 550B A55B (Reasoning) cwe-bench-aa:5b52def2-ac9b-4465-ad80-91ea8079e253config_7290235f7802366ba63fb8e54a91694fab50b54aa4f5f2c3972a4730210d3683 | 2% | Unknown | 1 |
| 19 | Qwen3.8 27B (Xhigh) cwe-bench-aa:b01dee41-c62b-48ed-8d16-984adc405e5cconfig_2e3dbb3ed500173be12ba8b8d88ab36215a87834ec2f22cca0882bb419c47abd | 0% | Unknown | 1 |
Source: Artificial Analysis
Explorer evidence
CyberGym-E2E-AA (Artificial Analysis)
Sources differ; shown separately, never averaged. Observed ; fresh until .
| Rank | Configuration | Capability | Uncertainty | Families |
|---|---|---|---|---|
| 1 | Kimi K3.1 Pro cybergym-e2e-aa:665c67ca-09eb-48f0-a7a3-0096e1d5de51config_9137d86891c84a18fb0f16404dc0f74127b8336cf5c2961cb470147ea83128d6 | 100% | Unknown | 1 |
| 2 | GPT-6 Sol (Daybreak Blue, max) cybergym-e2e-aa:93e12518-6e04-4713-b52d-2c56dac49c63config_21cb86828f24e05190c62a8852d17592073156f79d4999b118dd1246e55266db | 98% | Unknown | 1 |
| 3 | Mistral Large 4 (High) cybergym-e2e-aa:ea5019ae-b0b7-49a5-80b3-0e584807af0fconfig_75c18b592f6fa928cd9ab15c0646f2db69adf8e33c9bf12f76f5c2a29634f0ad | 84% | Unknown | 1 |
| 4 | MiMo-V2.6-Pro cybergym-e2e-aa:24d8fdfe-6241-424e-b7b7-1b6efb06e4fbconfig_d2d43e8d5689df753e6392c341337b2e6dd4762d22e91dad49ea093d782eba56 | 83% | Unknown | 1 |
| 5 | GPT-6 Luna (Max) cybergym-e2e-aa:6fb13851-40ce-4bda-ad3f-6b38e0c5daa7config_33ac70fd3b281ee171b6c00d09b2ba45cef22c00bc0f392b52048434150a7684 | 82% | Unknown | 1 |
| 6 | DeepSeek V4.1 Flash 1017 (Max) cybergym-e2e-aa:f151fd0b-2139-40cf-a3ec-995e8cedc592config_59c1432804e10864949fdfc2b3185c602e32478e624c871aea6db35db7e02d2f | 81% | Unknown | 1 |
| 7 | Step 5 Preview cybergym-e2e-aa:094ea3e3-ce41-419e-8bd1-723389663158config_546c5989e471b276bcbe01827827e77780566ed9cea3c284d0bb7463c70bf6f4 | 79% | Unknown | 1 |
| 8 | GLM-5.3-Flash cybergym-e2e-aa:19496b81-9f41-4214-a77a-1df803b3c5aeconfig_060b05cfcfdb2dabc451bb994cfc7d89be873e6ddbddce28b666b70efd4919fb | 78% | Unknown | 1 |
| 9 | Grok 4.7 (Xhigh) cybergym-e2e-aa:59627802-1546-422c-9a65-65b61c022d36config_aaf52d8b90c5731ebf5d37fa62c258a20f387ee2e22835179b6a4079821ebd5a | 78% | Unknown | 1 |
| 10 | Beam 1.0 cybergym-e2e-aa:69468cce-7b5c-414a-b498-41df4c34df85config_0bd4d1c6ee6f21292a3de2af5416528d7201738dcee267761032f95d701c33e3 | 48% | Unknown | 1 |
| 11 | GLM-5.3 (Max) cybergym-e2e-aa:cd684ea4-b475-4269-b001-d469d06d8a7aconfig_59bb674568f8034b73ea573ab86aa3f9e0a0974eddea4995d75181e368108a10 | 31% | Unknown | 1 |
| 12 | MiniMax-M3 cybergym-e2e-aa:277f939a-985b-4b37-859d-b3eabc7c0b26config_5d8cf94c70c5afe5499a6664d6343bae4c693746b626fc7f98daf2d4004a6ae1 | 26% | Unknown | 1 |
| 13 | Nemotron 3 Ultra 550B A55B (Reasoning) cybergym-e2e-aa:5b52def2-ac9b-4465-ad80-91ea8079e253config_9f839de5800c532334b7301b3e2525ae7ef08ad1936d86c2755628a628d7497f | 11% | Unknown | 1 |
| 14 | Gemini 3.8 Flash (High) cybergym-e2e-aa:8a999846-4c1d-4ce7-a8b7-1310a7166fd7config_9da903d7339742b5ed052cab92fa9a59c1b4a2141673ce00df237005d4559eb6 | 9% | Unknown | 1 |
| 15 | Claude Fable 5.1 (Max, Default Fallback) cybergym-e2e-aa:3e87c73e-a257-495e-9730-367a66229811config_905f4d29a1510e42332187295d138b93d0f88844c7cf2fa6e8c69c0fb1a1ac24 | 1% | Unknown | 1 |
| 16 | Claude Opus 5.5 (Max, Default Fallback) cybergym-e2e-aa:2f3c4dc9-a450-4303-8697-0237257cf08fconfig_be9d5d4cfd8c60b5507cbcba14b62fde7659214df695b4d9a56d22a296fac86c | 1% | Unknown | 1 |
| 17 | GPT-6 Astra (Max) cybergym-e2e-aa:2f339a97-9a0d-499a-9cb5-e0db665bfa25config_1cd15a6088e4a68c6b73ebd0232967c15d229fa1ee90e5f603cf1d781d24c0f3 | 0% | Unknown | 1 |
| 18 | Qwen3.8 2.4T A95B cybergym-e2e-aa:8df710d3-9dae-4498-9b4e-9818238e6f31config_6b8c60882bfe0cc2b68ce67c24c8182a9dcdc88e285e3e66fd5a2b5782dc76f6 | 0% | Unknown | 1 |
| 19 | Qwen3.8 27B (Xhigh) cybergym-e2e-aa:b01dee41-c62b-48ed-8d16-984adc405e5cconfig_bc187f45ff6ce8d22223ae39007257df7b2a2670c4dcf933a103b3ad40c2558b | 0% | Unknown | 1 |
Source: Artificial Analysis
Explorer evidence
DeepsecBench-AA (Artificial Analysis)
Sources differ; shown separately, never averaged. Observed ; fresh until .
| Rank | Configuration | Capability | Uncertainty | Families |
|---|---|---|---|---|
| 1 | GPT-6 Sol (Daybreak Blue, max) deepsecbench-aa:93e12518-6e04-4713-b52d-2c56dac49c63config_6b3c5e4139967db796ac1b932c7b35fbc294917d5911fd4ab09c2ff99dbaf9f6 | 100% | Unknown | 1 |
| 2 | GPT-6 Astra (Max) deepsecbench-aa:2f339a97-9a0d-499a-9cb5-e0db665bfa25config_4d9b933720cbb942b3ce8e62a6339d2fc1a934cc2886c88c83ce213d2932b597 | 76% | Unknown | 1 |
| 3 | Kimi K3.1 Pro deepsecbench-aa:665c67ca-09eb-48f0-a7a3-0096e1d5de51config_484e33155b0d79051491cc0de893c00de697a4e9451dd61179f53e66dfc4df4e | 76% | Unknown | 1 |
| 4 | Grok 4.7 (Xhigh) deepsecbench-aa:59627802-1546-422c-9a65-65b61c022d36config_831bccb5b1f4d3c61244297d49ac1330bfd97bab0ddf73b410b12c70ac24cdca | 55% | Unknown | 1 |
| 5 | Claude Opus 5.5 (Max, Default Fallback) deepsecbench-aa:2f3c4dc9-a450-4303-8697-0237257cf08fconfig_93873bfa5f379c9cf84ddafd82635b56167f772bbcfcb317e684b8c87ea70f3d | 55% | Unknown | 1 |
| 6 | MiMo-V2.6-Pro deepsecbench-aa:24d8fdfe-6241-424e-b7b7-1b6efb06e4fbconfig_d4d043bf707aaab71397052547a416529a08d6e16b739c8ea24087e6bd654e35 | 54% | Unknown | 1 |
| 7 | DeepSeek V4.1 Flash 1017 (Max) deepsecbench-aa:f151fd0b-2139-40cf-a3ec-995e8cedc592config_a6aac67405fb7f3a3f1629b02f60f9c31dec0cdd6706af76154ceb810884250b | 51% | Unknown | 1 |
| 8 | GLM-5.3 (Max) deepsecbench-aa:cd684ea4-b475-4269-b001-d469d06d8a7aconfig_0b2300dc53b9019c768961da18967120d4c054ec3c438c6fe19e9264f6aae2d3 | 49% | Unknown | 1 |
| 9 | GPT-6 Luna (Max) deepsecbench-aa:6fb13851-40ce-4bda-ad3f-6b38e0c5daa7config_40f58707d5621e7b3fd5a930856442c0b156820badbdea42a81d6c88a270a9a9 | 48% | Unknown | 1 |
| 10 | Claude Fable 5.1 (Max, Default Fallback) deepsecbench-aa:3e87c73e-a257-495e-9730-367a66229811config_adb39dd35b0fab85cac9cb9dec225f81ff4a0f47c2772e8d346802bbad63088f | 48% | Unknown | 1 |
| 11 | GLM-5.3-Flash deepsecbench-aa:19496b81-9f41-4214-a77a-1df803b3c5aeconfig_4a7373829e2784f35fc615103f058cc9ef795a9f49f28820ee28bac22d94ac2f | 40% | Unknown | 1 |
| 12 | Step 5 Preview deepsecbench-aa:094ea3e3-ce41-419e-8bd1-723389663158config_2ea19caff21ede7ec45f62bccbc156559dadfc85677a66b81e389e32099b5df0 | 31% | Unknown | 1 |
| 13 | Mistral Large 4 (High) deepsecbench-aa:ea5019ae-b0b7-49a5-80b3-0e584807af0fconfig_37c2736423c488bc368155e07d7ac2545d7f1345c77d6c0550b37975aede9c4a | 30% | Unknown | 1 |
| 14 | Gemini 3.8 Flash (High) deepsecbench-aa:8a999846-4c1d-4ce7-a8b7-1310a7166fd7config_ea7eac4b18f0b6869b68409457a9647c12b4868d8c59b627ace49396a5fd8c4c | 29% | Unknown | 1 |
| 15 | Qwen3.8 27B (Xhigh) deepsecbench-aa:b01dee41-c62b-48ed-8d16-984adc405e5cconfig_97991e5d00071c023c2aeca1aa27590564ae7ecdf572235d678b2e30dbe17404 | 22% | Unknown | 1 |
| 16 | Qwen3.8 2.4T A95B deepsecbench-aa:8df710d3-9dae-4498-9b4e-9818238e6f31config_28ef97e53dadeaffd8bf2b6d398cecfefec5d52628dd6f48468da2d819d61fa8 | 18% | Unknown | 1 |
| 17 | Beam 1.0 deepsecbench-aa:69468cce-7b5c-414a-b498-41df4c34df85config_22555ebb3170c6e7adf18b55bf668a5bd52d6b59bc33c691918eb821ff8d1435 | 12% | Unknown | 1 |
| 18 | MiniMax-M3 deepsecbench-aa:277f939a-985b-4b37-859d-b3eabc7c0b26config_714a4e54aa58e532b07b25e41b4c0cd4cf91abb8c681f333ac412a1acd87d6a3 | 0% | Unknown | 1 |
| 19 | Nemotron 3 Ultra 550B A55B (Reasoning) deepsecbench-aa:5b52def2-ac9b-4465-ad80-91ea8079e253config_4051bb5bdcf64e85605487923ef99a98cdb7243c56b3ca171d182dcc45eec753 | 0% | Unknown | 1 |
Source: Artificial Analysis
Explorer evidence
DeepsecBench-AA refusal rate (Artificial Analysis)
Sources differ; shown separately, never averaged. Observed ; fresh until .
| Rank | Configuration | Capability | Uncertainty | Families |
|---|---|---|---|---|
| 1 | Step 5 Preview deepsecbench-aa-refusal:094ea3e3-ce41-419e-8bd1-723389663158config_0d744c9349c45eee48558801a2e64788a25898dcf876bf3c3eb3d61c77204d4b | 100% | Unknown | 1 |
| 2 | Gemini 3.8 Flash (High) deepsecbench-aa-refusal:8a999846-4c1d-4ce7-a8b7-1310a7166fd7config_0dfa3cf1d51c5d92c3d2c39dcd046aa4ca1544f0ba358e7d61641256e9676bdb | 100% | Unknown | 1 |
| 3 | GPT-6 Luna (Max) deepsecbench-aa-refusal:6fb13851-40ce-4bda-ad3f-6b38e0c5daa7config_2781563178ea87f697b5ef1547353605d2791b615a8e30f9273b109440c7dd8d | 100% | Unknown | 1 |
| 4 | Beam 1.0 deepsecbench-aa-refusal:69468cce-7b5c-414a-b498-41df4c34df85config_33dae63bef612c8aa689232d76bd1f29345bb60d65d3b41e495580b1de36fb0b | 100% | Unknown | 1 |
| 5 | Claude Fable 5.1 (Max, Default Fallback) deepsecbench-aa-refusal:3e87c73e-a257-495e-9730-367a66229811config_3ce13e6f64478ae3785a8ef03d92466f1c60114a0557b3bede00c96287e3b30b | 100% | Unknown | 1 |
| 6 | DeepSeek V4.1 Flash 1017 (Max) deepsecbench-aa-refusal:f151fd0b-2139-40cf-a3ec-995e8cedc592config_3e3003dadfe981fd05d5f97c7d033e32e789c247cb91af3cb28ac8df13a606a4 | 100% | Unknown | 1 |
| 7 | GLM-5.3 (Max) deepsecbench-aa-refusal:cd684ea4-b475-4269-b001-d469d06d8a7aconfig_45ac5ec7fc62283ed01851f5e7762582f72e8fcf00b61e460c643f0d3cf0c52b | 100% | Unknown | 1 |
| 8 | GLM-5.3-Flash deepsecbench-aa-refusal:19496b81-9f41-4214-a77a-1df803b3c5aeconfig_48f76deb747ca9a5d71e4ac4622c5b6bb0486f4e02b47af68adf5e038bbfbfcc | 100% | Unknown | 1 |
| 9 | GPT-6 Sol (Daybreak Blue, max) deepsecbench-aa-refusal:93e12518-6e04-4713-b52d-2c56dac49c63config_6bdcd51cf95418186d41342afe0e2b386b4a7f2e1d4e03988f548cce5487d338 | 100% | Unknown | 1 |
| 10 | GPT-6 Astra (Max) deepsecbench-aa-refusal:2f339a97-9a0d-499a-9cb5-e0db665bfa25config_77e8f98ed607d567b5b390b16afb99648862dbfc382dd8041574c9f2f8d01b2a | 100% | Unknown | 1 |
| 11 | Grok 4.7 (Xhigh) deepsecbench-aa-refusal:59627802-1546-422c-9a65-65b61c022d36config_8d37fb5a1da6611a1c7282043af8e89647122e98ba802a52fc5c35569dc606d3 | 100% | Unknown | 1 |
| 12 | MiniMax-M3 deepsecbench-aa-refusal:277f939a-985b-4b37-859d-b3eabc7c0b26config_ae04d35b1e8a141aac01f35489fe47b67c5b0990805d661b11482ebc92f91ec1 | 100% | Unknown | 1 |
| 13 | Claude Opus 5.5 (Max, Default Fallback) deepsecbench-aa-refusal:2f3c4dc9-a450-4303-8697-0237257cf08fconfig_bb4777e156ea364dc9e60bf192e9c2114efc9840143170693825aa4d01cf343b | 100% | Unknown | 1 |
| 14 | MiMo-V2.6-Pro deepsecbench-aa-refusal:24d8fdfe-6241-424e-b7b7-1b6efb06e4fbconfig_c72d8c31d3c5709570f9b958fe41592a713c410086dd4c2d163491db12d0a9ff | 100% | Unknown | 1 |
| 15 | Mistral Large 4 (High) deepsecbench-aa-refusal:ea5019ae-b0b7-49a5-80b3-0e584807af0fconfig_e362b91a560b1a8128dfecdc87901e7a0057b620a5f66bdf56d9db9ed918ce4b | 100% | Unknown | 1 |
| 16 | Kimi K3.1 Pro deepsecbench-aa-refusal:665c67ca-09eb-48f0-a7a3-0096e1d5de51config_f4ea870d921eb091ec93c20ff4cfd1f9d510cded7930cc213239ba8de64d8b52 | 100% | Unknown | 1 |
| 17 | Nemotron 3 Ultra 550B A55B (Reasoning) deepsecbench-aa-refusal:5b52def2-ac9b-4465-ad80-91ea8079e253config_f85080543133be18f393b75596f4092260b68af65b1b03d2602fdff575610e68 | 100% | Unknown | 1 |
| 18 | Qwen3.8 27B (Xhigh) deepsecbench-aa-refusal:b01dee41-c62b-48ed-8d16-984adc405e5cconfig_330c0fde8078cdf099c7259327cd928aa671dff22a679cc6e53f660ecbf80129 | 62% | Unknown | 1 |
| 19 | Qwen3.8 2.4T A95B deepsecbench-aa-refusal:8df710d3-9dae-4498-9b4e-9818238e6f31config_3085ceffc621bb82821def04f62ca0f912778fe91ec524d96900a4cccfb7ae90 | 0% | Unknown | 1 |
Source: Artificial Analysis
Not enough independent evidence for a top set
- Calibration has not been validated
- Configuration overlap is below the required threshold
- Independent benchmark-family coverage is below the required threshold