GPT-5.6 Sol

Leaderboards

Where GPT-5.6 Sol sits among the top 20 in each category.

By task

Scored from votes on each kind of prompt alone, on the same 1,000-point scale.

Test results

Every test GPT-5.6 Sol has a published result on, out of 100%, with its rank among the models that took it.

• GPQA

  #2 / 256

  Graduate-level science questions written to be hard to look up.

  94.6%

  Self-reported

• MMMU-Pro

  #4 / 73

  College-level questions that need both an image and text to answer.

  83.0%

  Self-reported · In the vision score

• BrowseComp

  #4 / 68

  Finding hard-to-locate facts by browsing the web.

  90.4%

  Self-reported · In the agents score

• SWE-Bench Pro

  #10 / 61

  Harder, longer software engineering tasks from real codebases.

  64.6%

  Self-reported · In the coding score

• DeepSWE 1.1

  #15 / 50

  DeepSWE 1.1 evaluates software engineering agents using the mini-swe-agent harness.

  73.0%

  Self-reported · In the coding score

• Terminal-Bench 2.1

  #4 / 46

  Terminal-Bench 2.1 is an updated release of the Terminal-Bench benchmark that tests AI agents' ability to operate a computer via the terminal. It evaluates how well models handle real-world, end-to-end tasks autonomously, including compiling code, training models, setting up servers, system administration, data science workflows, and security tasks.

  88.8%

  Self-reported

• Toolathlon

  #12 / 42

  Tool Decathlon is a comprehensive benchmark for evaluating AI agents' ability to use multiple tools across diverse task categories. It measures proficiency in tool selection, sequencing, and execution across ten different tool-use scenarios.

  58.0%

  Self-reported · In the agents score

• MRCR v2 (8-needle)

  #4 / 32

  MRCR v2 (8-needle) is a variant of the Multi-Round Coreference Resolution benchmark that includes 8 needle items to retrieve from long contexts. This tests models' ability to simultaneously track and reason about multiple pieces of information across extended conversations.

  91.5%

  Self-reported · In the text score

• Terminal-Bench 4.0

  #16 / 28

  Terminal-Bench 4.0 evaluates AI agents on real-world work in terminal and command-line containerized environments across 66 tasks, with emphasis on science-adjacent and frontier engineering problems. Relative to earlier Terminal-Bench releases, 4.0 increases timeouts and adaptively raises RAM/CPU on selected tasks to reduce harness and resource confounds.

  37.3%

  Self-reported · In the coding score· In the agents score

• Agents' Last Exam

  #3 / 24

  Agents' Last Exam is a challenging benchmark for AI agents on hard, long-horizon tasks that test sustained reasoning, planning, and tool use, reported with and without tool access.

  52.7%

  Self-reported · In the agents score

• AutomationBench

  #20 / 24

  AutomationBench is a tool-use benchmark that evaluates AI agents on automating real-world workflows, testing their ability to orchestrate tools and complete multi-step automation tasks.

  18.1%

  Self-reported · In the agents score

• HealthBench Hard

  #3 / 24

  A challenging variation of HealthBench that evaluates large language models' performance and safety in healthcare through 5,000 multi-turn conversations with particularly rigorous evaluation criteria validated by 262 physicians from 60 countries

  33.1%

  Self-reported

• FrontierCode 1.1

  #8 / 23

  FrontierCode 1.1 evaluates whether coding-agent changes are mergeable, using unit tests, maintainer-defined rubrics, and verifiers. Runs flagged for unfair internet use receive a zero score.

  47.5%

  Self-reported · In the coding score

• HealthBench Professional

  #9 / 19

  HealthBench Professional evaluates model capability and safety for clinician use cases using real clinician-style chats and physician-authored grading rubrics.

  60.5%

  Self-reported

• FrontierMath

  #1 / 17

  A benchmark of hundreds of original, exceptionally challenging mathematics problems crafted and vetted by expert mathematicians, covering most major branches of modern mathematics from number theory and real analysis to algebraic geometry and category theory.

  89.0%

  Self-reported

• OSWorld 2.0

  #9 / 17

  OSWorld 2.0 is a benchmark of 108 long-horizon, real-world computer-use workflows spanning everyday and professional tasks. Each task is an end-to-end workflow that takes human users a median of about 1.6 hours, scored with a binary-completion metric, and targets challenges such as dynamic environments, cross-source reasoning, and implicit-state inference.

  62.6%

  Self-reported

• GDP.pdf

  #9 / 14

  GDP.pdf is a knowledge-work vision benchmark that evaluates models on economically valuable professional tasks presented as visual documents (PDFs), testing document-based reasoning, chart and table interpretation, and problem solving without tools.

  30.7%

  Self-reported

• HealthBench

  #8 / 14

  An open-source benchmark for measuring performance and safety of large language models in healthcare, consisting of 5,000 multi-turn conversations evaluated by 262 physicians using 48,562 unique rubric criteria across health contexts and behavioral dimensions

  57.0%

  Self-reported

• Artificial Analysis

  #1 / 13

  Artificial Analysis benchmark evaluates AI models across quality, speed, and pricing dimensions, providing a composite assessment of model capabilities for real-world usage.

  59.0%

  Self-reported

• DeepSWE

  #1 / 13

  DeepSWE is a software engineering agent benchmark evaluated with the mini-swe-agent harness, where each task is solved in an isolated container with no internet access. It measures an agent's ability to autonomously resolve real-world coding issues end to end.

  72.7%

  Self-reported

• Graphwalks BFS >128k

  #1 / 12

  A graph reasoning benchmark that evaluates language models' ability to perform breadth-first search (BFS) operations on graphs with context length over 128k tokens, testing long-context reasoning capabilities.

  90.7%

  Self-reported

• ExploitBench

  #3 / 8

  ExploitBench is a cybersecurity benchmark that evaluates a model's ability to discover and exploit software vulnerabilities, reported as the fraction of challenges where the model captures the target (Cap%).

  73.5%

  Self-reported

• ExploitGym

  #2 / 8

  ExploitGym is a large-scale, realistic benchmark built from real-world vulnerabilities across userspace programs, Google's V8 engine, and the Linux kernel. Given a vulnerability and a proof-of-vulnerability input, agents must craft a working end-to-end exploit that achieves unauthorized code execution.

  33.7%

  Self-reported

• BenchCAD

  #3 / 7

  BenchCAD is a benchmark for programmatic CAD reasoning built from 17,900 execution-verified CadQuery programs spanning 106 industrial part families, roughly half anchored to real ISO, DIN, EN, ASME, and IEC specification tables. It decomposes CAD capability into matched tasks; the Vision2Code task requires models to generate CadQuery code from multi-view renders, scored by voxel IoU against the reference geometry.

  70.6%

  Self-reported

• BenchCAD (with Python tool)

  #5 / 7

  BenchCAD variant evaluated with access to a Python tool for programmatic CAD reasoning.

  83.4%

  Self-reported

• SEC-bench Pro

  #2 / 7

  SEC-bench Pro is a self-evolving software-security benchmark that measures agent bug hunting on critical, high-complexity systems. It instantiates validated vulnerabilities across the V8 and SpiderMonkey JavaScript engines as reproducible vulnerability-discovery and proof-of-concept-generation tasks with oracle-based validation.

  71.2%

  Self-reported

• ARC-AGI-3

  #3 / 5

  ARC-AGI-3 is the third-generation Abstraction and Reasoning Corpus benchmark, an interactive-reasoning evaluation designed to measure fluid, novel problem-solving ability that remains far from saturated for frontier models.

  7.8%

  Self-reported

• FrontierMath Tier 4 (v2)

  #3 / 5

  FrontierMath Tier 4 subset from the v2 evaluation release.

  83.0%

  Self-reported

• HealthBench Consensus

  #2 / 5

  HealthBench Consensus is a HealthBench subset focused on questions where physician-created rubric criteria have especially high agreement, measuring healthcare performance and safety on consensus-evaluable conversations.

  95.5%

  Self-reported

• LifeSciBench

  #2 / 5

  LifeSciBench is an expert-authored, expert-reviewed benchmark of 750 open-ended life-science research tasks spanning seven workflows and seven biological domains. Responses are graded against 19,020 physician- and scientist-written rubric criteria rather than multiple-choice answers, and most tasks require interpreting attached artifacts such as figures, PDFs, and sequence files.

  59.9%

  Self-reported

• GeneBench-Pro

  #2 / 4

  GeneBench-Pro is a research-level benchmark of 129 multi-stage computational-biology problems spanning genomics, quantitative biology, and translational biomedicine. Each problem gives the agent a messy dataset, brief context, and a target estimand, and requires navigating dependent inferential decision points to reach a verifiable answer.

  28.7%

  Self-reported

• MMMU-Pro (with tools)

  #1 / 4

  MMMU-Pro variant evaluated with tool access enabled.

  84.6%

  Self-reported

• MRCR v2 (8-needle, 512K-1M)

  #2 / 4

  MRCR v2 8-needle variant evaluated on contexts from 512K to 1M tokens.

  73.8%

  Self-reported

• MedChemBench (Internal)

  #1 / 4

  MedChemBench is an internal OpenAI evaluation of medicinal-chemistry reasoning, testing whether models can support drug-discovery-relevant chemistry analysis and decision-making.

  48.3%

  Self-reported

• Big Finance Bench

  #1 / 3

  Big Finance Bench evaluates models on complex financial-analysis tasks that require retrieving and reasoning over financial documents and performing multi-step quantitative work.

  53.0%

  Self-reported

• Capture-the-Flag Challenges (Internal)

  #1 / 3

  Capture-the-Flag Challenges is OpenAI's internal expansion of competitive, professional-level cybersecurity capture-the-flag tasks used to evaluate vulnerability identification and exploitation capability under the Preparedness Framework.

  96.7%

  Self-reported

• Connectors

  #1 / 3

  Connectors is an OpenAI internal production benchmark measuring reliable use of connector-based tools in agentic workflows, reported as a pass rate.

  100.0%

  Self-reported

• Graphwalks BFS 1M

  #1 / 3

  GraphWalks BFS variant evaluated on 1M-token contexts.

  77.1%

  Self-reported

• Internal Research Debugging Evaluation

  #1 / 3

  The Internal Research Debugging Evaluation measures whether models can debug 41 real bugs from internal OpenAI research experiments (plus alignment-auditing tasks), where the original solutions took experienced researchers hours to days. Passing corresponds to providing assistance that would unblock the user, including partial root-cause explanations or fixes.

  68.3%

  Self-reported

• KernelGen 1P

  #1 / 3

  KernelGen 1P is an OpenAI AI-self-improvement evaluation that measures whether models can write and optimize compute kernels, part of the suite tracking progress toward accelerating internal research.

  61.1%

  Self-reported

• Management Consulting Tasks (Internal)

  #1 / 3

  Management Consulting Tasks is an internal OpenAI evaluation of long-horizon professional knowledge work drawn from management-consulting workflows, scoring whether models produce correct, decision-ready analyses.

  43.2%

  Self-reported

• NanoGPT

  #2 / 3

  NanoGPT is an OpenAI AI-self-improvement evaluation that measures whether models can optimize training recipes for small GPT-style models, part of the suite tracking progress toward accelerating internal research.

  9.7%

  Self-reported

• PostTrainBench Lite

  #2 / 3

  PostTrainBench Lite measures whether an agent can design and execute a full post-training strategy (data, prompts, RL recipe, and eval loop) for a pretrained base model under a constrained time budget, scored as normalized mean reward over the improvement window.

  50.3%

  Self-reported

• RSI Index

  #1 / 3

  The RSI (Recursive Self-Improvement) Index is OpenAI's aggregate metric across a bundle of internal AI-research evaluations, including debugging research systems, optimizing kernels and training recipes, and improving other models, measuring progress toward recursive self-improvement.

  57.9%

  Self-reported

• Search and Function-Calling

  #2 / 3

  Search and Function-Calling is an OpenAI internal production benchmark measuring reliable search-tool use and function calling in agentic workflows, reported as a pass rate.

  91.0%

  Self-reported