GPT-6.1 Sol

Leaderboards

Where GPT-6.1 Sol sits among the top 20 in each category.

By task

Scored from votes on each kind of prompt alone, on the same 1,000-point scale.

Test results

Every test GPT-6.1 Sol has a published result on, out of 100%, with its rank among the models that took it.

• MMMU-Pro

  #1 / 73

  College-level questions that need both an image and text to answer.

  86.0%

  Self-reported · In the vision score

• DeepSWE 1.1

  #8 / 50

  DeepSWE 1.1 evaluates software engineering agents using the mini-swe-agent harness.

  75.2%

  Self-reported · In the coding score

• HealthBench Hard

  #2 / 24

  A challenging variation of HealthBench that evaluates large language models' performance and safety in healthcare through 5,000 multi-turn conversations with particularly rigorous evaluation criteria validated by 262 physicians from 60 countries

  36.2%

  Self-reported

• FrontierCode 1.1

  #6 / 23

  FrontierCode 1.1 evaluates whether coding-agent changes are mergeable, using unit tests, maintainer-defined rubrics, and verifiers. Runs flagged for unfair internet use receive a zero score.

  50.2%

  Self-reported · In the coding score

• HealthBench Professional

  #5 / 19

  HealthBench Professional evaluates model capability and safety for clinician use cases using real clinician-style chats and physician-authored grading rubrics.

  64.2%

  Self-reported

• OSWorld 2.0

  #4 / 17

  OSWorld 2.0 is a benchmark of 108 long-horizon, real-world computer-use workflows spanning everyday and professional tasks. Each task is an end-to-end workflow that takes human users a median of about 1.6 hours, scored with a binary-completion metric, and targets challenges such as dynamic environments, cross-source reasoning, and implicit-state inference.

  71.4%

  Self-reported

• GDP.pdf

  #6 / 14

  GDP.pdf is a knowledge-work vision benchmark that evaluates models on economically valuable professional tasks presented as visual documents (PDFs), testing document-based reasoning, chart and table interpretation, and problem solving without tools.

  32.0%

  Self-reported

• HealthBench

  #5 / 14

  An open-source benchmark for measuring performance and safety of large language models in healthcare, consisting of 5,000 multi-turn conversations evaluated by 262 physicians using 48,562 unique rubric criteria across health contexts and behavioral dimensions

  58.5%

  Self-reported

• Humanity's Last Exam (no tools, text-only)

  #3 / 14

  Text-only Humanity's Last Exam variant evaluated without tool use.

  52.9%

  Self-reported

• APEX-Agents

  #1 / 11

  APEX-Agents is a benchmark evaluating AI agents on long horizon professional tasks that require sustained reasoning, planning, and execution across complex multi-step workflows.

  60.0%

  Self-reported

• Terminal-Bench-Science 0.1

  #5 / 7

  Terminal-Bench-Science 0.1 is a Stanford-led community benchmark of 70 tasks drawn from scientific research workflows across the life, physical, earth, mathematical, and engineering sciences. Tasks are authored and reviewed by scientists and researchers; scores are typically reported as accuracy with relatively large standard error due to strongly bimodal per-task outcomes.

  57.0%

  Self-reported

• AutomationBench v1.0.6

  #4 / 6

  AutomationBench v1.0.6 evaluates agent performance on multi-step workflow automation tasks.

  36.1%

  Self-reported

• FrontierMath Tier 4 (v2)

  #1 / 5

  FrontierMath Tier 4 subset from the v2 evaluation release.

  100.0%

  Self-reported

• HealthBench Consensus

  #1 / 5

  HealthBench Consensus is a HealthBench subset focused on questions where physician-created rubric criteria have especially high agreement, measuring healthcare performance and safety on consensus-evaluable conversations.

  96.0%

  Self-reported

• SimpleQA Verified

  #1 / 5

  SimpleQA Verified is a curated, reliability-focused subset of SimpleQA that addresses label noise and redundancy in the original benchmark, measuring short-form parametric factual accuracy of large language models on fact-seeking questions with single, indisputable answers.

  73.9%

  Self-reported

• FrontierCode 1.1 (Extended)

  #2 / 3

  FrontierCode 1.1 Extended subset: agentic coding tasks graded for mergeable patches, reported separately from the Main subset.

  60.4%

  Self-reported

• MentalHealthBench

  #1 / 1

  MentalHealthBench is an open benchmark developed with mental health experts for measuring how AI systems respond in realistic mental health conversations. It consists of 1,215 synthetic conversations covering diverse scenarios ranging from daily well-being topics to urgent mental health emergencies, scored with clinician-authored rubric criteria.

  57.9%

  Unverified