Hy4 preview

Leaderboards

Where Hy4 preview sits among the top 20 in each category.

By task

Scored from votes on each kind of prompt alone, on the same 1,000-point scale.

Test results

Every test Hy4 preview has a published result on, out of 100%, with its rank among the models that took it.

• GPQA

  #16 / 256

  Graduate-level science questions written to be hard to look up.

  92.3%

  Self-reported

• SWE-Bench Pro

  #7 / 61

  Harder, longer software engineering tasks from real codebases.

  65.7%

  Self-reported · In the coding score

• SWE-bench Multilingual

  #6 / 49

  A multilingual benchmark for issue resolving in software engineering that covers Java, TypeScript, JavaScript, Go, Rust, C, and C++. Contains 1,632 high-quality instances carefully annotated from 2,456 candidates by 68 expert annotators, designed to evaluate Large Language Models across diverse software ecosystems beyond Python.

  82.9%

  Self-reported

• Terminal-Bench 2.1

  #13 / 46

  Terminal-Bench 2.1 is an updated release of the Terminal-Bench benchmark that tests AI agents' ability to operate a computer via the terminal. It evaluates how well models handle real-world, end-to-end tasks autonomously, including compiling code, training models, setting up servers, system administration, data science workflows, and security tasks.

  85.4%

  Self-reported

• Toolathlon

  #3 / 42

  Tool Decathlon is a comprehensive benchmark for evaluating AI agents' ability to use multiple tools across diverse task categories. It measures proficiency in tool selection, sequencing, and execution across ten different tool-use scenarios.

  74.1%

  Self-reported · In the agents score

• MCP Atlas

  #5 / 38

  MCP Atlas is a benchmark for evaluating AI models on scaled tool use capabilities, measuring how well models can coordinate and utilize multiple tools across complex multi-step tasks.

  83.7%

  Self-reported

• NL2Repo

  #5 / 25

  NL2Repo evaluates long-horizon coding capabilities including repository-level understanding, where models must generate or modify code across entire repositories from natural language specifications.

  58.9%

  Self-reported · In the coding score

• Agents' Last Exam

  #24 / 24

  Agents' Last Exam is a challenging benchmark for AI agents on hard, long-horizon tasks that test sustained reasoning, planning, and tool use, reported with and without tool access.

  22.8%

  Self-reported · In the agents score

• AutomationBench

  #11 / 24

  AutomationBench is a tool-use benchmark that evaluates AI agents on automating real-world workflows, testing their ability to orchestrate tools and complete multi-step automation tasks.

  32.1%

  Self-reported · In the agents score

• CyberGym

  #14 / 21

  CyberGym is a benchmark for evaluating AI agents on cybersecurity tasks, testing their ability to identify vulnerabilities, perform security analysis, and complete security-related challenges in a controlled environment.

  78.4%

  Self-reported

• Program Bench

  #18 / 18

  Program Bench evaluates code-generation agents by asking them to recreate a program's behavior from only a compiled binary and documentation. It spans 200 tasks from small CLI tools to large systems such as FFmpeg and SQLite, with submissions judged against more than 248,000 fuzz-generated behavioral tests.

  17.5%

  Self-reported · In the coding score

• GDPval-AA

  #6 / 14

  GDPval-AA evaluates AI agents on economically valuable professional knowledge-work tasks and reports performance as an Elo score.

  55.9%

  Self-reported

• Humanity's Last Exam (no tools, text-only)

  #7 / 14

  Text-only Humanity's Last Exam variant evaluated without tool use.

  43.4%

  Self-reported

• WideSearch

  #1 / 14

  WideSearch is an agentic search benchmark that evaluates models' ability to perform broad, parallel search operations across multiple sources. It tests wide-coverage information retrieval and synthesis capabilities.

  83.9%

  Self-reported

• DeepSWE

  #5 / 13

  DeepSWE is a software engineering agent benchmark evaluated with the mini-swe-agent harness, where each task is solved in an isolated container with no internet access. It measures an agent's ability to autonomously resolve real-world coding issues end to end.

  64.3%

  Self-reported

• OfficeQA Pro

  #4 / 12

  OfficeQA Pro evaluates AI models on professional knowledge-work questions and tasks drawn from real office workflows, including document analysis, spreadsheet reasoning, and information synthesis across business domains.

  66.2%

  Self-reported

• APEX-Agents

  #4 / 11

  APEX-Agents is a benchmark evaluating AI agents on long horizon professional tasks that require sustained reasoning, planning, and execution across complex multi-step workflows.

  37.1%

  Self-reported

• MathArena Apex

  #3 / 11

  MathArena Apex is a challenging math contest benchmark featuring the most difficult mathematical problems designed to test advanced reasoning and problem-solving abilities of AI models. It focuses on olympiad-level mathematics and complex multi-step mathematical reasoning.

  74.2%

  Self-reported

• Humanity's Last Exam (with tools, text-only)

  #5 / 10

  Text-only Humanity's Last Exam variant evaluated with tool use enabled.

  55.4%

  Self-reported

• SWE-Marathon

  #8 / 10

  SWE-Marathon is an ultra-long-horizon software engineering benchmark covering tasks such as building compilers, optimizing kernels, and developing production-grade services. It measures whether agents can sustain quality across extremely long engineering trajectories.

  31.9%

  Self-reported

• SkillsBench

  #2 / 10

  SkillsBench evaluates coding agents on self-contained programming tasks, measuring practical engineering skills across diverse software development scenarios.

  62.9%

  Self-reported

• Job Bench

  #3 / 9

  Job Bench evaluates AI agents on realistic professional tasks that require multi-step planning, research, and production of work artifacts.

  61.7%

  Self-reported

• PostTrainBench

  #6 / 9

  PostTrainBench evaluates a model's ability to autonomously post-train base models. Given pretrain-only base models, the agent must complete the full pipeline of data synthesis, training, evaluation, and iteration within a time budget, scored across downstream benchmarks such as AIME2025, BFCL, GPQA Main, GSM8K, and HumanEval.

  35.6%

  Self-reported

• BioMysteryBench

  #4 / 7

  BioMysteryBench evaluates a model's ability to reason through challenging molecular biology problems, reporting performance on a hard subset and on the subset of problems solved by human experts.

  71.3%

  Self-reported

• CritPT

  #1 / 6

  CritPT is a challenging reasoning benchmark reported by Qwen for evaluating frontier mathematical and critical problem-solving capability.

  16.9%

  Self-reported

• ArXivMath

  #4 / 5

  ArXivMath is a final-answer benchmark of research-level mathematics maintained by MathArena. Problems are extracted monthly from recent arXiv paper abstracts, then filtered through automated and manual checks to ensure they are self-contained, non-trivial, and verifiable. Because problems are drawn from active research, the benchmark is more realistic and more closely connected to mathematical research than contest or olympiad benchmarks.

  66.6%

  Self-reported

• SuperChem

  #1 / 5

  SuperChem is a benchmark of advanced chemistry problems requiring expert-level domain knowledge and reasoning.

  66.4%

  Self-reported

• HorizonMath

  #1 / 4

  HorizonMath is an extremely difficult frontier mathematics benchmark designed to test the limits of mathematical reasoning on research-level and competition-beyond problems.

  8.8%

  Self-reported

• OneMillion Bench

  #3 / 4

  OneMillion Bench evaluates AI agents on high-economic-value tasks that require sustained, reliable execution across long-horizon real-world workflows.

  65.4%

  Self-reported

• Workspace Bench

  #2 / 4

  Workspace Bench evaluates AI agents on high-economic-value workplace tasks that span multi-step planning, file processing, and tool use across realistic office and productivity workflows.

  60.2%

  Self-reported

• BankerToolBench

  #1 / 3

  BankerToolBench is a public benchmark that evaluates models on banking and finance tool-use tasks. Models are scored against dataset rubrics, measuring their ability to correctly invoke tools and complete multi-step financial workflows.

  78.6%

  Self-reported

• DRACO

  #1 / 3

  DRACO is a deep research benchmark that evaluates an agent's ability to gather, synthesize, and reason over information to answer complex research questions. Scores are based on official rubrics per question, with the final score being the average across all questions.

  77.2%

  Self-reported