Hy3

Leaderboards

Where Hy3 sits among the top 20 in each category.

By task

Scored from votes on each kind of prompt alone, on the same 1,000-point scale.

Test results

Every test Hy3 has a published result on, out of 100%, with its rank among the models that took it.

• GPQA

  #25 / 256

  Graduate-level science questions written to be hard to look up.

  90.4%

  Self-reported

• SWE-Bench Verified

  #22 / 120

  Fixing real bugs in real GitHub projects, checked by the projects' own tests.

  78.0%

  Self-reported

• BrowseComp

  #15 / 68

  Finding hard-to-locate facts by browsing the web.

  84.2%

  Self-reported · In the agents score

• SWE-Bench Pro

  #27 / 61

  Harder, longer software engineering tasks from real codebases.

  57.9%

  Self-reported · In the coding score

• SWE-bench Multilingual

  #16 / 49

  A multilingual benchmark for issue resolving in software engineering that covers Java, TypeScript, JavaScript, Go, Rust, C, and C++. Contains 1,632 high-quality instances carefully annotated from 2,456 candidates by 68 expert annotators, designed to evaluate Large Language Models across diverse software ecosystems beyond Python.

  75.8%

  Self-reported

• Terminal-Bench 2.1

  #27 / 46

  Terminal-Bench 2.1 is an updated release of the Terminal-Bench benchmark that tests AI agents' ability to operate a computer via the terminal. It evaluates how well models handle real-world, end-to-end tasks autonomously, including compiling code, training models, setting up servers, system administration, data science workflows, and security tasks.

  71.7%

  Self-reported

• Toolathlon

  #26 / 42

  Tool Decathlon is a comprehensive benchmark for evaluating AI agents' ability to use multiple tools across diverse task categories. It measures proficiency in tool selection, sequencing, and execution across ten different tool-use scenarios.

  48.5%

  Self-reported · In the agents score

• MCP Atlas

  #10 / 38

  MCP Atlas is a benchmark for evaluating AI models on scaled tool use capabilities, measuring how well models can coordinate and utilize multiple tools across complex multi-step tasks.

  79.1%

  Self-reported

• AA-LCR

  #3 / 25

  Agent Arena Long Context Reasoning benchmark

  73.4%

  Self-reported

• NL2Repo

  #16 / 25

  NL2Repo evaluates long-horizon coding capabilities including repository-level understanding, where models must generate or modify code across entire repositories from natural language specifications.

  45.6%

  Self-reported · In the coding score

• IMO-AnswerBench

  #3 / 22

  IMO-AnswerBench is a benchmark for evaluating mathematical reasoning capabilities on International Mathematical Olympiad (IMO) problems, focusing on answer generation and verification.

  90.0%

  Self-reported

• Claw-Eval

  #4 / 14

  Claw-Eval tests real-world agentic task completion across complex multi-step scenarios, evaluating a model's ability to use tools, navigate environments, and complete end-to-end tasks autonomously.

  68.5%

  Self-reported

• Humanity's Last Exam (no tools, text-only)

  #4 / 14

  Text-only Humanity's Last Exam variant evaluated without tool use.

  47.0%

  Self-reported

• WideSearch

  #5 / 14

  WideSearch is an agentic search benchmark that evaluates models' ability to perform broad, parallel search operations across multiple sources. It tests wide-coverage information retrieval and synthesis capabilities.

  76.4%

  Self-reported

• DeepSWE

  #12 / 13

  DeepSWE is a software engineering agent benchmark evaluated with the mini-swe-agent harness, where each task is solved in an isolated container with no internet access. It measures an agent's ability to autonomously resolve real-world coding issues end to end.

  28.0%

  Self-reported

• APEX-Agents

  #11 / 11

  APEX-Agents is a benchmark evaluating AI agents on long horizon professional tasks that require sustained reasoning, planning, and execution across complex multi-step workflows.

  25.6%

  Self-reported

• MathArena Apex

  #7 / 11

  MathArena Apex is a challenging math contest benchmark featuring the most difficult mathematical problems designed to test advanced reasoning and problem-solving abilities of AI models. It focuses on olympiad-level mathematics and complex multi-step mathematical reasoning.

  38.7%

  Self-reported

• DeepSearchQA

  #4 / 10

  DeepSearchQA is a benchmark for evaluating deep search and question-answering capabilities, testing models' ability to perform multi-hop reasoning and information retrieval across complex knowledge domains.

  91.0%

  Self-reported

• Humanity's Last Exam (with tools, text-only)

  #6 / 10

  Text-only Humanity's Last Exam variant evaluated with tool use enabled.

  53.2%

  Self-reported

• SkillsBench

  #4 / 10

  SkillsBench evaluates coding agents on self-contained programming tasks, measuring practical engineering skills across diverse software development scenarios.

  55.3%

  Self-reported

• WildClawBench

  #4 / 6

  WildClawBench is an agentic coding benchmark from InternLM/Claw-Eval that reports overall model performance on real-world tool-using development tasks.

  53.6%

  Self-reported

• ArXivMath

  #5 / 5

  ArXivMath is a final-answer benchmark of research-level mathematics maintained by MathArena. Problems are extracted monthly from recent arXiv paper abstracts, then filtered through automated and manual checks to ensure they are self-contained, non-trivial, and verifiable. Because problems are drawn from active research, the benchmark is more realistic and more closely connected to mathematical research than contest or olympiad benchmarks.

  52.2%

  Self-reported

• FrontierScience Research

  #4 / 5

  FrontierScience Research is a benchmark evaluating AI models on cutting-edge scientific research questions requiring deep domain expertise, multi-step reasoning, and synthesis of complex scientific concepts across disciplines.

  21.3%

  Self-reported

• SuperChem

  #4 / 5

  SuperChem is a benchmark of advanced chemistry problems requiring expert-level domain knowledge and reasoning.

  54.9%

  Self-reported

• FrontierScience Olympiad

  #3 / 4

  FrontierScience Olympiad is a benchmark of olympiad-level scientific reasoning problems, testing expert understanding and multi-step reasoning across advanced natural-science domains.

  74.8%

  Self-reported

• HorizonMath

  #2 / 4

  HorizonMath is an extremely difficult frontier mathematics benchmark designed to test the limits of mathematical reasoning on research-level and competition-beyond problems.

  7.1%

  Self-reported

• CL-bench

  #1 / 3

  CL-bench is an open-source benchmark with its own data and rubrics for evaluating models on coding and agentic tasks, scored using a setup fully aligned with the official procedure.

  23.8%

  Self-reported

• PHYBench

  #1 / 3

  PHYBench is a benchmark of real-world physics problems spanning mechanics, electromagnetism, thermodynamics, optics, and modern physics, designed to evaluate physical perception and multi-step quantitative reasoning in large language models.

  77.4%

  Self-reported

• USAMO 2026

  #3 / 3

  USAMO 2026 evaluates models on the six problems from the 2026 United States of America Mathematical Olympiad, requiring rigorous proof-based reasoning. Following the MathArena methodology, proofs are graded against human expert rubrics by dual strong judge models, with the minimum of the two scores taken as final. Maximum score is 42 points.

  72.0%

  Self-reported

• CL-bench (Life)

  #1 / 1

  CL-bench Life variant.

  17.0%

  Unverified

• CMT-Benchmark

  #1 / 1

  CMT-Benchmark evaluates models on condensed matter theory problems, testing advanced physics reasoning across areas such as many-body systems, quantum field theory, and statistical mechanics.

  37.9%

  Unverified