MCP tool orchestration
Results are only compared within a matching group, so a model tested one way is never ranked against a model tested another way.
Coordinate MCP tools to complete a task
Ranking groups use separate scales. Compare configurations only inside the same group; EvalRank never flattens unlike systems into one leaderboard.
Cross-benchmark consensus
One ranking of models across this topic's admitted benchmarks, weighted the same way decisions are. Each score is the model's share of the best result.
| Rank | Model | Share of best | Benchmarks |
|---|---|---|---|
| 1 | GPT-6 Sol | 100.0% | 1 |
| 2 | GPT-5.6 Sol | 97.2% | 2 |
| 3 | Claude Sonnet 5.5 | 94.9% | 1 |
| 4 | Claude Haiku 5.5 | 74.0% | 1 |
| 5 | Claude Sonnet 5 | 70.2% | 1 |
| 6 | Claude Sonnet 4.6 | 64.2% | 2 |
| 7 | claude-haiku-4-5 | 34.8% | 2 |
Agent system · Agentic
Exploratory - not a ranked winner yetrg-mcp-tool-orchestration-agent-system-agentic-agent-system-v1Explorer evidence
AutomationBench
Sources differ; shown separately, never averaged. Observed ; fresh until .
| Rank | Configuration | Capability | Uncertainty | Families |
|---|---|---|---|---|
| 1 | Gemini 4 Carbon (High) automationbench:Gemini 4 Carbon (High)config_e84fd0f5e59e3b8b964a542fe8b56ecab0908e9f39f2d70113f5958cfa087e47 | 100% | Unknown | 1 |
| 2 | Gemini 4 Carbon (Medium) automationbench:Gemini 4 Carbon (Medium)config_a2c8a23afe15a50ed53869a323929661fea1b94adf65bc273c11b4020ed5df2a | 98% | Unknown | 1 |
| 3 | Gemini 4 Argon (High) automationbench:Gemini 4 Argon (High)config_5d2ea9047d28198e8f9d59299190aa9c7aa87d2bd564c2115aa0707b1f5d5876 | 95% | Unknown | 1 |
| 4 | Gemini 4 Argon (Medium) automationbench:Gemini 4 Argon (Medium)config_75cc09df9b6c3d3d3e18e6383c21bc35c08dd06ca09b77f568d02e92dfd2182d | 93% | Unknown | 1 |
| 5 | Claude Sonnet 5.5 (default fallbacks, Max) automationbench:Claude Sonnet 5.5 (default fallbacks, Max)config_24df20a13eb0b425ca163f80c22071b19068661c34ded9d3b9e5705968516640 | 83% | Unknown | 1 |
| 6 | Claude Opus 5.5 (default fallbacks, Max) automationbench:Claude Opus 5.5 (default fallbacks, Max)config_7d573508be54b283cb5b9da147bca47fbbe0b1999ac7fb96b2188a220b9e089d | 79% | Unknown | 1 |
| 7 | GPT-6 Astra (Max) automationbench:GPT-6 Astra (Max)config_a111367f67b8b5bd55e3f1ba436f00f5ad0576721d4f7d1e6c4143e11c591bb5 | 77% | Unknown | 1 |
| 8 | GPT-6 Astra (XHigh) automationbench:GPT-6 Astra (XHigh)config_1f8e8f6eec27fbfec971a587be15a5f5ff62ee7697d7d822c2958b926c2ff34e | 72% | Unknown | 1 |
| 9 | GPT-6 Astra (High) automationbench:GPT-6 Astra (High)config_790ab0f3fbf9884524df53588326d812de8d04023ac15a466f32581d4e65192c | 69% | Unknown | 1 |
| 10 | Claude Sonnet 5.5 (default fallbacks, XHigh) automationbench:Claude Sonnet 5.5 (default fallbacks, XHigh)config_38411a32bd75fcf1e407e91922f08b0fce52b9b484b9245cc728060bdae2438d | 68% | Unknown | 1 |
| 11 | Claude Opus 5.5 (default fallbacks, XHigh) automationbench:Claude Opus 5.5 (default fallbacks, XHigh)config_d156e9fba8f9e0c88442fbdd4c718e9be1cf24a73d7dae582bf97215d95028f5 | 66% | Unknown | 1 |
| 12 | GPT-6 Astra (Medium) automationbench:GPT-6 Astra (Medium)config_926a2bc702bb0bb28519705811f091560eb26aa4a9bb8be86ff75cac1d16a489 | 63% | Unknown | 1 |
| 13 | GPT-6 Sol (XHigh) automationbench:GPT-6 Sol (XHigh)config_bcd7cba856ee883231d52bd67acbc695263f9d9b6d30f84eaeace38956512f45 | 62% | Unknown | 1 |
| 14 | Claude Opus 5.5 (default fallbacks, High) automationbench:Claude Opus 5.5 (default fallbacks, High)config_0ebb36576624ae257dc2e45c695763430113c68d0d679619c79e9a4354157e8e | 61% | Unknown | 1 |
| 15 | GPT-6 Sol (Max) automationbench:GPT-6 Sol (Max)config_0ae6fc70172dbbe05e35a00fc5dd331f291406d91e47eea51030184315c38fb2 | 59% | Unknown | 1 |
| 16 | Gemini 4 Carbon (Low) automationbench:Gemini 4 Carbon (Low)config_0022478ede674058059e374503c9b6e6bcf75d2a928e33fd243a27ffd8624647 | 58% | Unknown | 1 |
| 17 | GPT-6 Sol (High) automationbench:GPT-6 Sol (High)config_d91199fe2cf3c7cd2c612bd66f5f8e17b5e0d2c0f8693d73d8ca14f15c3ee3f1 | 58% | Unknown | 1 |
| 18 | Gemini 3.7 Flash (High) automationbench:Gemini 3.7 Flash (High)config_c0cb9d036f2945081943c28d51317f9d58b56a8414efe3190d392ddf789049a1 | 56% | Unknown | 1 |
| 19 | GPT-6 Astra (Low) automationbench:GPT-6 Astra (Low)config_bbddcff9f0e33fbae7e6300bccdbfcd194fb626b52162cb7076b546464af9d6f | 56% | Unknown | 1 |
| 20 | Gemini 3.8 Flash (Medium) automationbench:Gemini 3.8 Flash (Medium)config_095b858200422ae2a40f4de8999825fc1169b7634be76069bcf1630b1c43b24c | 55% | Unknown | 1 |
| 21 | Gemini 3.8 Flash (High) automationbench:Gemini 3.8 Flash (High)config_6fb0bae145783ee3b353b0ca54772a85f9a0286483bbeb389733f86938808322 | 55% | Unknown | 1 |
| 22 | Claude Opus 5.5 (default fallbacks, Medium) automationbench:Claude Opus 5.5 (default fallbacks, Medium)config_80d1f934366a7bb43b583184ba8e185bc373d803668d1a190a1eedf251603a85 | 55% | Unknown | 1 |
| 23 | Claude Sonnet 5.5 (default fallbacks, High) automationbench:Claude Sonnet 5.5 (default fallbacks, High)config_d63c3d216dfc81713b0aad7a4df86eecf61dac1a3b70f7d6533ee2c968b88775 | 55% | Unknown | 1 |
| 24 | GPT-5.6 Sol (Max) automationbench:GPT-5.6 Sol (Max)config_940a9985646aa2382eb4b6250e00321dbd4848b72f450cb4f78d19d7dbb641ec | 53% | Unknown | 1 |
| 25 | Claude Opus 5 (Max) automationbench:Claude Opus 5 (Max)config_6c30ed4edc61c38064a7dfb3e0367ca8c4cdb51598fe51220192c028246cc3f7 | 50% | Unknown | 1 |
| 26 | GPT-6 Sol (Medium) automationbench:GPT-6 Sol (Medium)config_8991690c485ea7dcfea93958c8eba8f59139a26a7986edbf7874bce3350c00e2 | 50% | Unknown | 1 |
| 27 | GPT-5.6 Sol (XHigh) automationbench:GPT-5.6 Sol (XHigh)config_6914105bab6b84c5f7857a2daf10cece7fcb4d8dddd505c553f87f034e308fc7 | 49% | Unknown | 1 |
| 28 | Claude Opus 5 (XHigh) automationbench:Claude Opus 5 (XHigh)config_379a60c5001bd6e347dcb9bba5e3b43039bb97f4873a41c6a9cef13654870ccf | 47% | Unknown | 1 |
| 29 | GPT-6 Astra (None) automationbench:GPT-6 Astra (None)config_e0b5ab861bbb87d5302dfdcbf53d1cdbd54665212c0848975a0be745fc54e14e | 47% | Unknown | 1 |
| 30 | GPT-5.6 Sol (High) automationbench:GPT-5.6 Sol (High)config_48d7667275573b7f84817bffb5ec6ec29dacc0cab92c3c5221335df4c4bb778e | 46% | Unknown | 1 |
| 31 | Claude Opus 5.5 (default fallbacks, Low) automationbench:Claude Opus 5.5 (default fallbacks, Low)config_800d2388f620e5a5f20eef6c3d8634c88201853f5fad8a2fad124433dd44f85d | 45% | Unknown | 1 |
| 32 | Claude Opus 5 (Medium) automationbench:Claude Opus 5 (Medium)config_38af236b2377faa1da2903ec87e4e8f14bcf8b96ff6dd9f2fffacf69b30e9c01 | 44% | Unknown | 1 |
| 33 | GPT-5.6 Terra (Max) automationbench:GPT-5.6 Terra (Max)config_66ab823f13f439e05e8d8001792d1c4c28d3ec56bfe2dd9d3bf63dd1407d2cfd | 44% | Unknown | 1 |
| 34 | Claude Sonnet 5.5 (default fallbacks, Medium) automationbench:Claude Sonnet 5.5 (default fallbacks, Medium)config_3560eaf095efa2d9a30365236cec46651041d20016cbf77cf4616c66f6b134ba | 43% | Unknown | 1 |
| 35 | Kimi K3 automationbench:Kimi K3config_7fe334a4ba41ed28eb6687619f824748ebd7d4cbe9e99cbbc796d00233f7ff2b | 42% | Unknown | 1 |
| 36 | Claude Fable 5.1 (Max) automationbench:Claude Fable 5.1 (Max)config_5a05961823ba6f70a3e2e768a7f45bfd863c22abfd6cd358f58566ed229d2c83 | 42% | Unknown | 1 |
| 37 | Claude Fable 5.1 (XHigh) automationbench:Claude Fable 5.1 (XHigh)config_7f4f479be11ddadd0afd2f3cfbb3488555fed2308346fe2e0749498ba7c6cd7d | 40% | Unknown | 1 |
| 38 | GPT-6 Sol (Low) automationbench:GPT-6 Sol (Low)config_1bd5e38efb3ba147b60840c08f238ca2f71746d64ea30531aa0693e53c4a938e | 39% | Unknown | 1 |
| 39 | Muse Spark 1.3 (Max) automationbench:Muse Spark 1.3 (Max)config_843ac9cf1d323628644076897150b6dbb112c5dc07c08c5a953d78dba9e6df0d | 38% | Unknown | 1 |
| 40 | GPT-6 Luna (Max) automationbench:GPT-6 Luna (Max)config_b76316b06957619144608fa6606d170063ba394dd5f95b0f5749ae897bca6673 | 38% | Unknown | 1 |
| 41 | Claude Opus 5 (High) automationbench:Claude Opus 5 (High)config_ee7ea30c3c59725eadd7a39119f3a97212ec84a86ab3f654f8d64c73e8e2786c | 38% | Unknown | 1 |
| 42 | Claude Opus 5 (Low) automationbench:Claude Opus 5 (Low)config_b4fd15298892317616ae4d34fd5dd4ab6048d96d82232bdaee2783bc4e4fba1a | 38% | Unknown | 1 |
| 43 | GPT-5.6 Sol (Medium) automationbench:GPT-5.6 Sol (Medium)config_c72ba89a8964da6640cd2110b783cdfedb335585b95cd3c6b91d0deab5c8f2e8 | 36% | Unknown | 1 |
| 44 | Claude Sonnet 5.5 (default fallbacks, Between tools) automationbench:Claude Sonnet 5.5 (default fallbacks, Between tools)config_dd8b98785baf8f3f1260e2caa1f113add821b2615aee2ff6c405b4865a9372ea | 34% | Unknown | 1 |
| 45 | Claude Sonnet 5.5 (default fallbacks, Low) automationbench:Claude Sonnet 5.5 (default fallbacks, Low)config_035c383289bf40bf0aa358453f36f0328f095913f12bc4ad3cd1569b33571835 | 34% | Unknown | 1 |
| 46 | DeepSeek V4 Flash (Max) automationbench:DeepSeek V4 Flash (Max)config_8adec4fa3f4445669babf6791fd0c8b1d13faaf33fe2e1b5ab77a8578a4b8ae1 | 34% | Unknown | 1 |
| 47 | Claude Fable 5.0 (Max) automationbench:Claude Fable 5.0 (Max)config_842a0156d3bc02b07e5557c6f76d8817f36c71821facb520fbdc89c69ff7a0c8 | 32% | Unknown | 1 |
| 48 | GPT-5.6 Luna (Max) automationbench:GPT-5.6 Luna (Max)config_9f407c81e3e732a87ee9966b14be9fed65ccbfd04e208da88a597efc81348ffa | 32% | Unknown | 1 |
| 49 | Gemini 3.6 Flash (High) automationbench:Gemini 3.6 Flash (High)config_edbc6630272a0a2a0d726c090798ce08f4aefc75f0768687904fe6d52bae8ef1 | 32% | Unknown | 1 |
| 50 | Claude Opus 4.8 (XHigh) automationbench:Claude Opus 4.8 (XHigh)config_57a659c9e83d821868dab27ab0c159c57b5c97fb718ac66d09d1c5fd5f6f6f87 | 31% | Unknown | 1 |
| 51 | GPT-5.5 (XHigh) automationbench:GPT-5.5 (XHigh)config_a44921f0a694e40f05bbaf53e494a883740be650b703bdeb2e6d88247204cc91 | 31% | Unknown | 1 |
| 52 | Claude Opus 4.8 (Max) automationbench:Claude Opus 4.8 (Max)config_6fc288bc8171bcb11a4d115a1236362d6f2c1cace393b9455376b1a7446a6bd0 | 31% | Unknown | 1 |
| 53 | Gemini 3.6 Flash (Medium) automationbench:Gemini 3.6 Flash (Medium)config_9883ff967e78a4bc23fc63fa7515b224b645585ffa07f0adb4793b3e321de2a0 | 31% | Unknown | 1 |
| 54 | Claude Fable 5.0 (High) automationbench:Claude Fable 5.0 (High)config_1e09d3f93509113695dc72682b6d4bd1a220ac6f8973c216ab0268c5ac6e85d3 | 30% | Unknown | 1 |
| 55 | Claude Opus 4.8 (High) automationbench:Claude Opus 4.8 (High)config_7af054dd8b90178d41fa5ba45ac6b85a7bd9843dcaecf1792dae961b753dc779 | 30% | Unknown | 1 |
| 56 | Gemini 3.6 Flash (Low) automationbench:Gemini 3.6 Flash (Low)config_96db8eff7c7487fa8b3edcc5a5b6749f8a26631031b5987c4fab2f292049e971 | 29% | Unknown | 1 |
| 57 | Claude Fable 5.0 (XHigh) automationbench:Claude Fable 5.0 (XHigh)config_9e1bb3935c837e86f05fbcdb05fbce0a02636a323176f9ee7dd106488e279d29 | 28% | Unknown | 1 |
| 58 | GPT-5.6 Terra (XHigh) automationbench:GPT-5.6 Terra (XHigh)config_0938f8c812774ae0f3301e7e73f7da3d97672ad7ad16a0e5328e641a5a91c82c | 27% | Unknown | 1 |
| 59 | GPT-6 Luna (High) automationbench:GPT-6 Luna (High)config_bca94f1f9298e81fe8ba14b154d2a09ce9a1508b6ee4861dd5df64ad82bfd7e4 | 27% | Unknown | 1 |
| 60 | Claude Opus 4.8 (Medium) automationbench:Claude Opus 4.8 (Medium)config_9bc17cc37c8ee20959203a8f4947c5b3af8bc2609a8396087cdbaa58c01d45af | 26% | Unknown | 1 |
| 61 | Claude Fable 5.0 (Medium) automationbench:Claude Fable 5.0 (Medium)config_016ff96a0e4331eec7742f8a9d4fd190c38091cc7b4e5bc176cd0f3d68a1b0ee | 25% | Unknown | 1 |
| 62 | Claude Opus 4.8 (Low) automationbench:Claude Opus 4.8 (Low)config_1933e3ab343725efb7037ee8c4dc8cb0825f9decf507e06c6690479aeb34940f | 25% | Unknown | 1 |
| 63 | Claude Opus 4.7 (Max) automationbench:Claude Opus 4.7 (Max)config_59e570755a4cf5d7fbb07f086d044c1bbf95cfa2dbf629a504a5ca5ef00d9ddf | 25% | Unknown | 1 |
| 64 | Gemini 3.5 Flash (High) automationbench:Gemini 3.5 Flash (High)config_755449f4b0a662664c5dbafb0c1d0dbeb840e0293ef0ca007b51d69549ca9561 | 25% | Unknown | 1 |
| 65 | GPT-5.6 Luna (XHigh) automationbench:GPT-5.6 Luna (XHigh)config_b8209af48673e711712a4e514aa2a6cfa4d0b1f8fee6c10d13dc8ac9dd117e30 | 24% | Unknown | 1 |
| 66 | GPT-5.5 (High) automationbench:GPT-5.5 (High)config_8eff39f6fbfb5661ca1f0191233bafda3a85e8b642e4070bea355b251519831b | 24% | Unknown | 1 |
| 67 | GPT-6 Luna (XHigh) automationbench:GPT-6 Luna (XHigh)config_e34d98131c28ce73764be376543465683a40ccd737a87830ffec3e5937ce96be | 23% | Unknown | 1 |
| 68 | Claude Fable 5.0 (Low) automationbench:Claude Fable 5.0 (Low)config_538922a0f6a51bf97e124666493ed18ea7f0329ba5ad951d9cb396bcbc209790 | 23% | Unknown | 1 |
| 69 | Claude Haiku 5.5 (default fallbacks, Max) automationbench:Claude Haiku 5.5 (default fallbacks, Max)config_2d730b70968a8445ec73c8964d1d773d21bfdca141a8aecd61edc914ef388709 | 22% | Unknown | 1 |
| 70 | GPT-5.6 Terra (High) automationbench:GPT-5.6 Terra (High)config_683e62a9dd0be7c3ce9e8da6b7a1fcf7591ada7d4dd9e0206f7fe96677b1f9d2 | 22% | Unknown | 1 |
| 71 | GPT-5.6 Sol (Low) automationbench:GPT-5.6 Sol (Low)config_98207ee200454b8ea84e956f71ecba3113d7f36d19749b06da86c646b055fc01 | 22% | Unknown | 1 |
| 72 | GPT-5.4 (XHigh) automationbench:GPT-5.4 (XHigh)config_e8064f3e3a4c1e8d61ed769cf52efb0672f7816498084d140131ffa26193195a | 21% | Unknown | 1 |
| 73 | Gemini 3.5 Flash (Medium) automationbench:Gemini 3.5 Flash (Medium)config_26955bb84631e73a688c108a6155adfdc11dd856fe5c262f1d36057bb7b3542a | 21% | Unknown | 1 |
| 74 | Claude Sonnet 5 (Max) automationbench:Claude Sonnet 5 (Max)config_972f27eeef575bb98d84efd9cfa4a1f3d1159d8210431dc30a14b7070ff906b9 | 20% | Unknown | 1 |
| 75 | Claude Haiku 5.5 (default fallbacks, High) automationbench:Claude Haiku 5.5 (default fallbacks, High)config_e7d0cc2bffe97d13bcd84dc6661c180b1b596bfb9511ead4345d21bb132d36c6 | 19% | Unknown | 1 |
| 76 | GPT-5.4 (High) automationbench:GPT-5.4 (High)config_f9a4242ec20cdc9f97ad292ac4ac3ad675cbfa489dd1392154e088019786c3b3 | 19% | Unknown | 1 |
| 77 | Claude Sonnet 5 (High) automationbench:Claude Sonnet 5 (High)config_20282f5cf0dbd7c112d93547824df5fcdee2cde63ea002164dd4701f113cf642 | 18% | Unknown | 1 |
| 78 | Claude Haiku 5.5 (default fallbacks, XHigh) automationbench:Claude Haiku 5.5 (default fallbacks, XHigh)config_3554b3b0c77b31f9b29d994dc9b7dec9286af6e127eca2d7b9b21e74a9f60978 | 18% | Unknown | 1 |
| 79 | Claude Opus 4.7 (XHigh) automationbench:Claude Opus 4.7 (XHigh)config_4358b0b845f5bdb1fc0fe74e682a7e6f8611b28ef6e9020d7cfc79b317926dc8 | 18% | Unknown | 1 |
| 80 | GPT-6 Luna (Medium) automationbench:GPT-6 Luna (Medium)config_aaed56f1623126e9e0f3c9f83fe8fbf57f088abe7a8ea052159749bea11aacb9 | 18% | Unknown | 1 |
| 81 | GPT-5.6 Luna (High) automationbench:GPT-5.6 Luna (High)config_6d2cc4180e12e117508f5272e4612a8a92945c4b96779a26a8f0ae0becd12c3c | 17% | Unknown | 1 |
| 82 | Claude Opus 4.7 (High) automationbench:Claude Opus 4.7 (High)config_de903b17293f654071e06d06bbd68685a6b8b1daf141a1536c8828e4506ef7df | 17% | Unknown | 1 |
| 83 | GPT-6 Sol (None) automationbench:GPT-6 Sol (None)config_0b6eb43d309326b5f507fb96fd383a58ccf83eb6b8d0a617de9be5081c7d0de7 | 17% | Unknown | 1 |
| 84 | Gemini 3.5 Flash (Low) automationbench:Gemini 3.5 Flash (Low)config_231bbc69d64c7ecdc426b5ddca802275a979a51fc9c79074919b3122df0802c9 | 17% | Unknown | 1 |
| 85 | GPT-5.5 (Medium) automationbench:GPT-5.5 (Medium)config_656eec9a117375a7a1d64d5d93991d50d899a608624f25c56a5d2c89818e4c16 | 17% | Unknown | 1 |
| 86 | Gemini 3.1 Pro (preview) (High) automationbench:Gemini 3.1 Pro (preview) (High)config_cfdc83f500f5f2027e4b83edc59e0701a1c000bb9645a20874170899130e1dd7 | 16% | Unknown | 1 |
| 87 | Claude Haiku 5.5 (default fallbacks, Medium) automationbench:Claude Haiku 5.5 (default fallbacks, Medium)config_9f8d37e89ebdeb515dc7d6ec7db89a53eedd3872a4414d8abda399b9f7fcc557 | 16% | Unknown | 1 |
| 88 | GPT-5.6 Terra (Medium) automationbench:GPT-5.6 Terra (Medium)config_f4ab754d979d37b2482d7f87e1438f5495ac5188a8cfeb8c4d1f4fceec5bdf3a | 15% | Unknown | 1 |
| 89 | GLM 5.2 (Max) automationbench:GLM 5.2 (Max)config_0c13ca0ba185e27a9f5c82afde759268a5f5e77f5594464c0103d3957d614a99 | 14% | Unknown | 1 |
| 90 | Claude Sonnet 5 (XHigh) automationbench:Claude Sonnet 5 (XHigh)config_46217d1aac5ae584b99d820b707bf9a7ced9949c46d9b327d5de4da4df08aab0 | 14% | Unknown | 1 |
| 91 | GPT-5.6 Terra (Low) automationbench:GPT-5.6 Terra (Low)config_617e8371d486c79f25c0e07ae993e3495c113b65fc930005ee07b38fb44deec6 | 14% | Unknown | 1 |
| 92 | Gemini 3.1 Pro (preview) (Medium) automationbench:Gemini 3.1 Pro (preview) (Medium)config_c8fd43f830cbdb9b4503da89b1db1ec30caa30f3879ebae54294322105660a5b | 13% | Unknown | 1 |
| 93 | Claude Opus 4.7 (Medium) automationbench:Claude Opus 4.7 (Medium)config_2800b26b583f7d5d02436aae781338e2a96e54e82429d8191e16676583b5de1a | 13% | Unknown | 1 |
| 94 | GLM 5.1 (Max) automationbench:GLM 5.1 (Max)config_55964024fb11ee6532d7554782d28f013f1056dbf08ebb8c9bd447189e6cfb73 | 12% | Unknown | 1 |
| 95 | Claude Haiku 5.5 (default fallbacks, Low) automationbench:Claude Haiku 5.5 (default fallbacks, Low)config_34ae6300e749bf6a300ffb78bd3ee3c48e0a47d5c09df05e1daa18a8777d97bd | 11% | Unknown | 1 |
| 96 | Claude Sonnet 5 (Medium) automationbench:Claude Sonnet 5 (Medium)config_69f2227cb119fd66a3ec31face79684824168194551dc155b990cebd7ed9f33d | 11% | Unknown | 1 |
| 97 | Claude Opus 4.7 (Low) automationbench:Claude Opus 4.7 (Low)config_0ace5839fcad4044d3df332bae99e48417cbbf6744e139c5c78b4e135428ccf1 | 10% | Unknown | 1 |
| 98 | Claude Sonnet 4.6 (Max) automationbench:Claude Sonnet 4.6 (Max)config_b98e2fee1d3cf0c74de948a3d0d45b5b80c9f51a16a7f5d660c823a1cb292226 | 10% | Unknown | 1 |
| 99 | GPT-5.4 (Medium) automationbench:GPT-5.4 (Medium)config_d1966837232621d34f8fc04ca0a4b7e80b692c23a077aa498455ead06822bd81 | 10% | Unknown | 1 |
| 100 | Claude Sonnet 4.6 (High) automationbench:Claude Sonnet 4.6 (High)config_917f2ef386a3a6346baf40eb79c1e64adcd06b4922ac984800b0ce54f1171f51 | 9% | Unknown | 1 |
| 101 | Minimax M3 (Max) automationbench:Minimax M3 (Max)config_0b34173771cf7e524bf382430c1f0bddec294a39a446c29f1d2838f536d6f2c9 | 9% | Unknown | 1 |
| 102 | Gemini 3.1 Pro (preview) (Low) automationbench:Gemini 3.1 Pro (preview) (Low)config_9c231cb0712e622189c62e9d503de44ce2e978d4bb9970b0b1cdcaa60df629f6 | 8% | Unknown | 1 |
| 103 | Qwen 3.6+ (High) automationbench:Qwen 3.6+ (High)config_ac8c95d3767eb2450487c2de81bb2dd53fee30be7c5348cdad6aaa714dd0f03d | 8% | Unknown | 1 |
| 104 | GPT-5.6 Luna (Medium) automationbench:GPT-5.6 Luna (Medium)config_7a9ec1209ea915c2f075866e5c9e7e496774b0e4098e9cecadfa82fc82247085 | 8% | Unknown | 1 |
| 105 | Gemini 3.5 Flash Lite (Medium) automationbench:Gemini 3.5 Flash Lite (Medium)config_5965182fe9b7ac2b4b4bb68e3a68f28e12fa95580a161dff9e32b8cbbb17b45e | 7% | Unknown | 1 |
| 106 | GPT-5.6 Sol automationbench:GPT-5.6 Solconfig_0fe3df83d5405cb879e7c888d3ec219c37f8abfb1b77fac891c4d1a83bec7ef2 | 7% | Unknown | 1 |
| 107 | Qwen 3.7+ automationbench:Qwen 3.7+config_b1ab5e4e13101409eda689ce8df061d2b2c0d934660398fa80ddea334e3250a2 | 7% | Unknown | 1 |
| 108 | Claude Sonnet 4.6 (Medium) automationbench:Claude Sonnet 4.6 (Medium)config_60e7088d0f31383097dd602e14d9c3293c966054d0a1a86a59362b9a01100b83 | 7% | Unknown | 1 |
| 109 | Claude Sonnet 5 (Low) automationbench:Claude Sonnet 5 (Low)config_d512e3d3de66475a32922dd7c1fcea8ff173d6c57fb3126d644cafbcd373d3db | 7% | Unknown | 1 |
| 110 | Gemini 3.5 Flash Lite (High) automationbench:Gemini 3.5 Flash Lite (High)config_a1f5d3b72d69f9325d23335544f0cb70a1b4ef6115866fd4f4e2ebfbe10ba194 | 6% | Unknown | 1 |
| 111 | Kimi K2.7 Code automationbench:Kimi K2.7 Codeconfig_ab845b1f3ab15c72ebe2f75b680321bc084498e7d4c7e7fdf618ce0d0e9df307 | 6% | Unknown | 1 |
| 112 | Claude Sonnet 4.6 (Low) automationbench:Claude Sonnet 4.6 (Low)config_0ccde52aa509afd4e2350cb38a101a54b7665b73a20654c5a17a66aaa7271c05 | 5% | Unknown | 1 |
| 113 | GPT-5.5 (Low) automationbench:GPT-5.5 (Low)config_24e8d53c0136d8b5629ecff77e4455c68b17328ff9f8296451a1cbd32c3bfc0b | 5% | Unknown | 1 |
| 114 | Kimi K2.6 automationbench:Kimi K2.6config_f459a46c670d91cee6f393e92df639704024fd609fd9772213160ed6b2f0382b | 5% | Unknown | 1 |
| 115 | GPT-5.6 Luna (Low) automationbench:GPT-5.6 Luna (Low)config_f3f80ec5f6bcba5acda6376458367925ac1495556eb56c1f3de677a3f4b14d56 | 3% | Unknown | 1 |
| 116 | Gemma 4 31B (Max) automationbench:Gemma 4 31B (Max)config_76f665c645a8364c51556168df78bf0eea23138b7a2b5da4677778f80647fdd4 | 3% | Unknown | 1 |
| 117 | GPT-5.4 (Low) automationbench:GPT-5.4 (Low)config_4fb251b393d11b9386f207b0f37d67759be0dcaed5ecbcbcb0b77896ba4606e5 | 3% | Unknown | 1 |
| 118 | GPT-6 Luna (Low) automationbench:GPT-6 Luna (Low)config_4570dbe7a5543ef8a8248ae5c61dc2ebee528e2938861a3dab8c36e9c93250fe | 2% | Unknown | 1 |
| 119 | Gemini 3.5 Flash (Minimal) automationbench:Gemini 3.5 Flash (Minimal)config_b54c11ea31d7647b3d7db5177afe3a157affea280a4d973fb311e13a5ac35fb4 | 2% | Unknown | 1 |
| 120 | GPT-5.5 automationbench:GPT-5.5config_11e954943fd9ad77cabe35d72b06200fa98db7508157154de7c3785df6605897 | 2% | Unknown | 1 |
| 121 | GPT-5.6 Terra automationbench:GPT-5.6 Terraconfig_cf03f5b3c4e7a90d8a90130659a125a2377620b1d4b57ce89ccd6d5b184c9c6a | 1% | Unknown | 1 |
| 122 | Claude Haiku 4.5 automationbench:Claude Haiku 4.5config_99e5bf5740dd30017b10fe9e3fe1e4ea8edb3f10d66879ee9bbfa545d4f1ef14 | 1% | Unknown | 1 |
| 123 | Minimax M2.7 (High) automationbench:Minimax M2.7 (High)config_bea74616b427750ea86b97daee9f4973d8894f2958d5cb66a35740ab3e41eb0c | 1% | Unknown | 1 |
| 124 | GPT-6 Luna (None) automationbench:GPT-6 Luna (None)config_415054aa14c4ebcb5967ebb9c853eb09f6f5951d2b3334f17783a5d0fa7e19cc | 1% | Unknown | 1 |
| 125 | GPT-5.4 automationbench:GPT-5.4config_5471863f8d76f77203a971b328349f3a41a8c9c6b3fb7f436e38250d2bcd80a0 | 0% | Unknown | 1 |
| 126 | GPT-5.6 Luna automationbench:GPT-5.6 Lunaconfig_bc7a49651846ff7d1b19f8a048bc5bf5f309f59aea304331361eb342ea17a0af | 0% | Unknown | 1 |
| 127 | GPT-OSS 120B (High) automationbench:GPT-OSS 120B (High)config_cfa46f246cd66e02d2b0dd618d8de1f72d1e999d10abe5661068575dfbb636a4 | 0% | Unknown | 1 |
| 128 | Gemini 3.5 Flash Lite (Low) automationbench:Gemini 3.5 Flash Lite (Low)config_f9644e1833ae570913681da98d4ba6f799add0287f54495d4696bc9704bd2858 | 0% | Unknown | 1 |
Explorer evidence
MCP-Atlas
Sources differ; shown separately, never averaged. Observed ; fresh until .
| Rank | Configuration | Capability | Uncertainty | Families |
|---|---|---|---|---|
| 1 | Muse Spark 1.1 mcp-atlas:Muse Spark 1.1config_865e6ef688acad27a266666361d822a7e12a1942834149471cf90eb044a5cc19 | 100% | Unknown | 1 |
| 2 | claude-Fable 5.1 mcp-atlas:claude-Fable 5.1config_f328e3998ed1b45103e1649d2a56eb49ab52c4778c1c7fbb601e4a09c6aeaa84 | 98% | Unknown | 1 |
| 3 | claude-opus-5 mcp-atlas:claude-opus-5 | xhighconfig_3b3da0e8d86c658bad97c185bb9a483596e0071070d7cf422d13e9cc9fe0b05a | 95% | Unknown | 1 |
| 4 | Qwen3.8-2.4T-A95B mcp-atlas:Qwen3.8-2.4T-A95B | xhighconfig_1390b81eca2217ffd224a50bf934dcc42987b13c9d626c93327d94eb0f77af78 | 92% | Unknown | 1 |
| 5 | GLM 5.3 mcp-atlas:GLM 5.3config_4230f907d1dac8530ad295039e60329f300e8b5c03fbdce4de791a419dedd214 | 92% | Unknown | 1 |
| 6 | Gemini 3.5 Flash mcp-atlas:Gemini 3.5 Flash | highconfig_b5619ab8c5a5e56ed4fb07192538b4c60398967a98bd1e84aa003fd40b5e4701 | 91% | Unknown | 1 |
| 7 | Claude Fable 5 mcp-atlas:Claude Fable 5config_0af42acc3be8b535fe45e8528a110115e3dd4285b0a8d6240ec15d79e7e0a663 | 90% | Unknown | 1 |
| 8 | kimi-k3 mcp-atlas:kimi-k3 | maxconfig_6f65c5cb596ddad3df6a940b2b8290cd820b7461521faa187b025da51bb12bbc | 88% | Unknown | 1 |
| 9 | Muse Spark mcp-atlas:Muse Sparkconfig_47028847b57705edc5e8485a9c561b1f1088e683eee397c8f02e5aca140727c7 | 88% | Unknown | 1 |
| 10 | claude-opus-4-8 mcp-atlas:claude-opus-4-8 | maxconfig_56ece83507303ffc5a54b226e22f8386a83c748a7904b50812c53e8ad254d616 | 88% | Unknown | 1 |
| 11 | gpt-5.6-sol mcp-atlas:gpt-5.6-solconfig_14e638ae9ebf6ea3d9836679b084df5917ab75b83fc026d187f1b5fc48b4a1ce | 87% | Unknown | 1 |
| 12 | Inkling-small mcp-atlas:Inkling-smallconfig_f75f6d17e3c366b14091c9625505c7c70b115a94c65fc825da57566170ece92b | 81% | Unknown | 1 |
| 13 | claude-opus-4-7 mcp-atlas:claude-opus-4-7 | maxconfig_05a06b37c0a4784f1186582e4860942edb8279ac8f2c9a12392cfa5971436ecf | 81% | Unknown | 1 |
| 14 | gemini-3.1-pro-preview mcp-atlas:gemini-3.1-pro-preview | highconfig_d16487ba4140f90ab8e629b2bd15a10683a8a0b38a5b422addc585163fa8fc2b | 79% | Unknown | 1 |
| 15 | glm-5p2 mcp-atlas:glm-5p2config_9699b17040d7a7e000ce022b080619085249b28ef899665d8db4775a89f737fc | 78% | Unknown | 1 |
| 16 | claude-opus-4-6 mcp-atlas:claude-opus-4-6 | maxconfig_07f910f0cb8559b0faf9b23dcb787e5fdbc4c3f879f396903e7bd95686c37c80 | 76% | Unknown | 1 |
| 17 | Inkling mcp-atlas:Inkling | xhighconfig_40527c8c703ea129249d2e9b7e67687c3ef8098f305b5caec0730ef571e53d5d | 75% | Unknown | 1 |
| 18 | glm-5p1 mcp-atlas:glm-5p1config_b052e66e22d1ee4d2bad78bd6113209e193d9488c4c8167aa1d24acfb8e5813c | 74% | Unknown | 1 |
| 19 | gpt-5.5 mcp-atlas:gpt-5.5 | xhighconfig_f5077b4a3b6a10c5fb4ceb2f793cfdc2cac38a5eb5a489a8a3c9aac250172ca6 | 73% | Unknown | 1 |
| 20 | gpt-5.4 mcp-atlas:gpt-5.4 | xhighconfig_0f3ec8178be2beb726efa9e54d9bbe8b0ebe163b046451b1c0e618665453b129 | 63% | Unknown | 1 |
| 21 | gemini-3-pro-preview mcp-atlas:gemini-3-pro-previewconfig_83f12c7c544421b5ca5d1de95594c62fae3c76cf7eccba162e4debc2541c8e64 | 63% | Unknown | 1 |
| 22 | claude-opus-4-5 mcp-atlas:claude-opus-4-5 | highconfig_3554bcad8ceadb2c86d5eef891263f8fd1aa56bae525786b724d3f03c2d2c253 | 62% | Unknown | 1 |
| 23 | claude-sonnet-4-6 mcp-atlas:claude-sonnet-4-6config_4b53a75c0bf72e6e78ceb7f3710a73def172cecfd1481e27fcd95a9c19efbcf9 | 61% | Unknown | 1 |
| 24 | gpt-5.2 mcp-atlas:gpt-5.2 | xhighconfig_880a3ad28d5bd24cc745c1e6bf260f3d3d196263b0d8cdf285f6e5956e605818 | 57% | Unknown | 1 |
| 25 | kimi-k2p5 mcp-atlas:kimi-k2p5config_fba0ee6e849a5c93b689b9d0e3b26034b06b7a2cf9ee2222d6dd7bae9ed30c23 | 51% | Unknown | 1 |
| 26 | Nemotron 3 Ultra mcp-atlas:Nemotron 3 Ultra | thinkingconfig_90a7dd9d9a4e3539f3d56132efa93b8433610735bd9985371cdf6cb9d9c90879 | 48% | Unknown | 1 |
| 27 | gemini-3-flash-preview mcp-atlas:gemini-3-flash-previewconfig_c55ef9457cb3ed93270f0f5fd9bde56665511b7d532751c17944fee26cfff272 | 46% | Unknown | 1 |
| 28 | claude-sonnet-4-5 mcp-atlas:claude-sonnet-4-5 | thinkingconfig_6bff94df52f572fd58c975cffac1e3f415ea59f18493e9d86d141a7a4eb0b839 | 40% | Unknown | 1 |
| 29 | glm-4p7 mcp-atlas:glm-4p7config_4f4efb7aa85ec2fa61d79749a4c2219af212f5fd4df9f0b743c3ac807193597b | 37% | Unknown | 1 |
| 30 | gemini-3.1-flash-lite mcp-atlas:gemini-3.1-flash-lite | highconfig_42937da441ce651eb4640549352dc34c2ff35e1c9d9aa539e3fab35e9a28ad9d | 35% | Unknown | 1 |
| 31 | gpt-5.4-mini mcp-atlas:gpt-5.4-mini | xhighconfig_5131da2f10f8f911dabcc4468860eda25d94ced53ea86759194bac58c9236b14 | 34% | Unknown | 1 |
| 32 | gpt-5.1 mcp-atlas:gpt-5.1 | highconfig_5cc4d6801b3c13b8d9033fac9aa87f665bd0a7383b61f77e8fbb0c1de9892ed9 | 21% | Unknown | 1 |
| 33 | o3-pro mcp-atlas:o3-proconfig_e8fdd869d71f2e1bc97d8ea15b2b753bc9b579a1aad2976d16370aca6735df0c | 9% | Unknown | 1 |
| 34 | claude-haiku-4-5 mcp-atlas:claude-haiku-4-5config_0980c78f9c8c89f5a9484df46ff794b35518b66cbdb0a81dce709b25a7594c9d | 0% | Unknown | 1 |
Explorer evidence
tau3-bench Banking Knowledge
Sources differ; shown separately, never averaged. Observed ; fresh until .
| Rank | Configuration | Capability | Uncertainty | Families |
|---|---|---|---|---|
| 1 | Qwen 3.8 Max qwen3-8-max_sierra_2026-08-04config_a26a8d10f6a1f8c55b2ae39ab85459ab0c65f1c000c357819e67050c18dcc2fc | 55% | Unknown | 1 |
| 2 | Claude Opus 5 claude-opus-5_sierra_2026-08-04config_47e4a643a1c879897e35acc19a52271766e0d24716490d433f47f1250db796f5 | 49% | Unknown | 1 |
| 3 | Grok 4.5 grok-4-5_sierra_2026-08-04config_f4ada0ce1a4b8539b6b16eb70ecd5937576c5612b57153c438fab65dc2644d96 | 48% | Unknown | 1 |
| 4 | GPT-5.6-sol gpt-5-6-sol_sierra_2026-08-04config_a3711cbeeacfc54666ebdd477195ebcaac4642a7aaff878de8a40aa19416a568 | 47% | Unknown | 1 |
| 5 | GPT-5.5 gpt-5-5_sierra_2026-05-05config_ad5041576a055588650eb979cd38244d3c92d872d052d82dafb205bfa74cad3d | 45% | Unknown | 1 |
| 6 | Muse Spark 1.1 muse-spark-1-1_sierra_2026-08-04config_9e9d0ff85e658a60ac616eb6536f77d57ceb67178820283b459b21fc91e37164 | 40% | Unknown | 1 |
| 7 | Claude Opus 4.7 claude-opus-4-7_sierra_2026-05-05config_442ebf6d5800bf504193b385c4554a54f85fd95ef7845c2c54708c68861e0216 | 40% | Unknown | 1 |
| 8 | Claude Opus 4.8 claude-opus-4-8_sierra_2026-08-04config_53dbefe80b9536d86ae8cb24809c569e4c07be1cca283343280e6be8bb563145 | 40% | Unknown | 1 |
| 9 | Claude Fable 5 claude-fable-5_sierra_2026-08-04config_7bf49a074c69d34202fcdb4efe6bd1931566fb4e29b9f6e773de585366ecc67b | 40% | Unknown | 1 |
| 10 | GPT-5.4 gpt-5-4_sierra_2026-03-25config_81277ce76a2443b26ca7f3db93d65526b5c66a673148fbe228d16db598379eb7 | 39% | Unknown | 1 |
| 11 | Kimi K3 kimi-k3_sierra_2026-08-04config_0ac9cb2463f2a18ab6613660534e55deb75861bf471271f6aac39d02e4fbc4f8 | 37% | Unknown | 1 |
| 12 | GLM-5.2 glm-5-2_sierra_2026-08-04config_ebd02a3bcc5de5fdc15841e5214662ff73054096aefd3b9db9de3c1ced3cc679 | 37% | Unknown | 1 |
| 13 | GPT-5.2 gpt-5-2_sierra_2026-02-26config_0a61de28f7e96ed842d0988f4ff17d52f778051137caca3b12c8809fab3a1e24 | 32% | Unknown | 1 |
| 14 | Distyl ButtonAgent distyl-buttonagent_distyl_2026-03-25config_91e6242dafc69cf6d5b045dfa3e30dbca4c38c586b61a1ecbb28962e03b9d7c6 | 31% | Unknown | 1 |
| 15 | Claude Opus 4.6 claude-opus-4-6_sierra_2026-05-05config_79e8f1ddf8bcb22e452c0322f0a78c5fbcba059bfa5acc9acce62e5702eee472 | 27% | Unknown | 1 |
| 16 | Gemini 3 Flash gemini-3-flash_sierra_2026-03-02config_8095fa144e2c9e0944193c86724a27a3b80ddc5fb3d62d561fa298fa54c79701 | 27% | Unknown | 1 |
| 17 | Gemini 3.1 Pro Preview gemini-3-1-pro-preview_sierra_2026-05-05config_03d738b190a028db163266b395b4eca0e0f719f7bb43bd1a6db305d10f91fe2d | 26% | Unknown | 1 |
| 18 | Claude Sonnet 4.5 claude-sonnet-4-5_sierra_2026-02-26config_12ecf4a9f0ae1e2f9815530d97599aef123564d64fddecf8fd39aad2165b1a6e | 25% | Unknown | 1 |
| 19 | Inkling inkling_sierra_2026-08-04config_4a49384540eefc80eaff83393407119b8902549d0f6a45bed272b60bd4ee4937 | 25% | Unknown | 1 |
| 20 | Claude Opus 4.5 claude-opus-4-5_sierra_2026-02-26config_311ca40d213a79344ba2b618fe7f29fcf71ff8cbaaa8cab632499a1bf6dc01b0 | 25% | Unknown | 1 |
| 21 | Grok 4.2 grok-4-2_sierra_2026-05-05config_62cd4f1736a5056e6cf24954b6e0fe45123ac500dcbe4726080840da176bc5a8 | 18% | Unknown | 1 |
| 22 | Gemini 3 Pro gemini-3-pro_sierra_2026-03-02config_66ed191ca632aad42f3b0c528e2912ccef29a49ac134594c4f0209ac45e77ef7 | 18% | Unknown | 1 |
| 23 | Grok 4 fast grok-4-fast_sierra_2026-05-05config_3835bdd87e33611308716fcc185637e9d15136981937b1d92c25d7ae29775927 | 16% | Unknown | 1 |
| 24 | Gemini 2.5 Pro gemini-2-5-pro_sierra_2026-05-05config_5908158689798bbd956792b26fbe7a5a831a7907a7f811a7e974b7231763d00b | 14% | Unknown | 1 |
| 25 | Grok 4.1 fast grok-4-1-fast_sierra_2026-05-05config_320872c79fbebdc15ccb655f4f54fd38971fc9049a5bdb2a2ce5209a2fc755ea | 13% | Unknown | 1 |
| 26 | GPT-5.2 gpt-5-2-none_sierra_2026-02-26config_414bd51eab2ed44504e875d40d7cfe5b9246c80ca5bb91ca01accabf5625ea97 | 13% | Unknown | 1 |
| 27 | GLM-5 glm-5-think_sierra_2026-03-02config_3f1ad20f11e4674c490744ee115c4bc0c88728936353a52de2d451677b63812f | 10% | Unknown | 1 |
| 28 | Qwen3.5-397B-A17B qwen3.5-397b-a17b-think_sierra_2026-03-02config_54cb0b0fed47395e14ff29324863729b3a12e1f93ad171469ba3b588e56dd177 | 10% | Unknown | 1 |
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
- tau3-banking-knowledge benchmark evidence
Explorer evidence
Toolathlon
Sources differ; shown separately, never averaged. Observed ; fresh until .
| Rank | Configuration | Capability | Uncertainty | Families |
|---|---|---|---|---|
| 1 | GLM 5.3 Flash toolathlon-verified:GLM 5.3 Flash | Default | reasoning=maxconfig_a1aadfcb56405a0d0eb191f5135a9d9e2ccb7dee2c238801790214226a6a78ba | 78% | Unknown | 1 |
| 2 | Kimi K3 toolathlon-verified:Kimi K3 | Default | reasoning=maxconfig_3c68f7e1a7f2486c9ff1b6472b4c236cc5336d62caaede33a34a94c8b18131ba | 77% | Unknown | 1 |
| 3 | Claude Opus 4.8 toolathlon-verified:Claude Opus 4.8 | Default | reasoning=maxconfig_40bbb60a17931a21d62e2e9759806e9ee686d1024058b5e441fa991ef3511175 | 76% | Unknown | 1 |
| 4 | Muse Spark 1.2 toolathlon-verified:Muse Spark 1.2 | Default | reasoning=xhighconfig_9feb5dad2c8a17db8a8a081b11fc4f030ecd8b0058bcdf81d49c2fc78dcf1f0a | 76% | Unknown | 1 |
| 5 | Muse Spark 1.1 toolathlon-verified:Muse Spark 1.1 | Default | reasoning=xhighconfig_b3dd3426004e01579b0fa24d5e2d3af6a1055eea5369d7be8cb7a3bbbc76785f | 76% | Unknown | 1 |
| 6 | DeepSeek V4 Pro 0813 toolathlon-verified:DeepSeek V4 Pro 0813 | Default | reasoning=maxconfig_a442fb1aa154afa78f31f9333e14fbcceeb68d81bc8ff09f12cbf0a76563a7b7 | 74% | Unknown | 1 |
| 7 | GPT-5.5 toolathlon-verified:GPT-5.5 | Default | reasoning=xhighconfig_14a00cacba7d65e688a5c760163b8a5c43f157cd0ac3c74a8908ada4bf75af4a | 74% | Unknown | 1 |
| 8 | Claude Sonnet 5 toolathlon-verified:Claude Sonnet 5 | Default | reasoning=maxconfig_1e92cd632b85b5cae4db98f0747a379d6971e1901e817b6c70ba668a1d4f2581 | 72% | Unknown | 1 |
| 9 | DeepSeek V4 Flash 0731 toolathlon-verified:DeepSeek V4 Flash 0731 | Default | reasoning=maxconfig_088be908339ec2c5d18efb1d89f10f2a589d84061a7df91df36136544658b0fe | 71% | Unknown | 1 |
| 10 | Gemini 3.5 Flash toolathlon-verified:Gemini 3.5 Flash | Default | reasoning=highconfig_7424c21364f1c26da7c03d4246703a4574f36bcf742ec051d026f1a2866292a5 | 67% | Unknown | 1 |
| 11 | Gemini 3.1 Pro toolathlon-verified:Gemini 3.1 Pro | Default | reasoning=highconfig_6387719edef57bba35460d38ec2768197f35735c9d27dfbc49a4dacf8b8749c4 | 61% | Unknown | 1 |
| 12 | GLM 5.2 toolathlon-verified:GLM 5.2 | Default | reasoning=maxconfig_32d1f9502ae274db028b789ff6ea67f263fbea181b3f45dd6ab9a33c40321138 | 60% | Unknown | 1 |
| 13 | Kimi K2.7 Code toolathlon-verified:Kimi K2.7 Code | Defaultconfig_013440bcabc69495c6ad1bfdd9fbf082239f7167ccc9dc6f3a721aacb6144341 | 58% | Unknown | 1 |
| 14 | Kimi K2.6 toolathlon-verified:Kimi K2.6 | Defaultconfig_c6aa2be8da38947cda5c3c1ab5ce5ba57e8dd5b1a105e0734ec8b3d785cf6e9a | 58% | Unknown | 1 |
| 15 | Gemini 3.5 Flash-Lite toolathlon-verified:Gemini 3.5 Flash-Lite | Default | reasoning=highconfig_425f38f920acec6ada6502aa651d29309804f062d7ed03c8eed3c54fa62c02d5 | 57% | Unknown | 1 |
| 16 | Hy3 toolathlon-verified:Hy3 | Default | reasoning=highconfig_446911b3349e7d3b5aad2cacc1d0b27500170fc47e6e2a36c433ae9723f807b4 | 57% | Unknown | 1 |
| 17 | DeepSeek V4 Pro toolathlon-verified:DeepSeek V4 Pro | Default | reasoning=maxconfig_7527b100f7599e56734b703f9af93a6879ba5d71dd9e6ee4c136a08140172023 | 56% | Unknown | 1 |
| 18 | Inkling-Small toolathlon-verified:Inkling-Small | Default | reasoning=xhighconfig_2ed2bf4206c9be5f348242ccb5526dc4a6b0195f657a6b33e34f7be422924e5d | 54% | Unknown | 1 |
| 19 | DeepSeek V4 Flash toolathlon-verified:DeepSeek V4 Flash | Default | reasoning=maxconfig_f8a8eb20465f68c1be43c3ad9d4e9c2d41a80a4425f90ea4071fd5ba0355acf8 | 51% | Unknown | 1 |
| 20 | MiMo V2.5 toolathlon-verified:MiMo V2.5 | Defaultconfig_5fbd46f4fa4376922f5b7517a6d938344057a9542a0428d8b0f62cef6a1b2f2d | 49% | Unknown | 1 |
| 21 | MiniMax M2.7 toolathlon-verified:MiniMax M2.7 | Defaultconfig_60f8d3600b9bcb5b8877cbd43ef898f2662079d65b4a7db91dc9319f8507dee6 | 48% | Unknown | 1 |
| 22 | Inkling toolathlon-verified:Inkling | Default | reasoning=xhighconfig_cb28eb8804eaba125c2e9fce39868f34be7b2e47fb2da8261a4f54b3f73d5427 | 46% | Unknown | 1 |
| 23 | Qwen3.5 397B-A17B toolathlon-verified:Qwen3.5 397B-A17B | Defaultconfig_69d341d1f12b8f7988cf1cbd0f72515cb5fab5ddcb250c6ce72a038de5a66e00 | 41% | Unknown | 1 |
| 24 | Nemotron 3 Ultra toolathlon-verified:Nemotron 3 Ultra | Defaultconfig_e3ea78145d4d26cc32da136c704897302f33b6b7cb7d7beba53fcc66f7be9c5e | 34% | Unknown | 1 |
| 25 | Kimi K2.5 toolathlon-verified:Kimi K2.5 | Defaultconfig_8be3ad7082404c320dce37e31ba992fae9c1f78835fc94cef5b7b4e2504672a5 | 33% | Unknown | 1 |
Not enough independent evidence for a top set
- Calibration has not been validated
- Configuration overlap is below the required threshold
- Independent benchmark-family coverage is below the required threshold