Claude Opus 4.8

Leaderboards

Where Claude Opus 4.8 sits among the top 20 in each category.

By task

Scored from votes on each kind of prompt alone, on the same 1,000-point scale.

Test results

Every test Claude Opus 4.8 has a published result on, out of 100%, with its rank among the models that took it.

• GPQA

  #6 / 256

  Graduate-level science questions written to be hard to look up.

  93.6%

  Self-reported

• SWE-Bench Verified

  #4 / 120

  Fixing real bugs in real GitHub projects, checked by the projects' own tests.

  88.6%

  Self-reported

• Humanity's Last Exam

  #9 / 108

  Humanity's Last Exam (HLE) is a multi-modal academic benchmark with 2,500 questions across mathematics, humanities, and natural sciences, designed to test LLM capabilities at the frontier of human knowledge with unambiguous, verifiable solutions

  57.9%

  Self-reported · In the text score· In the vision score

• BrowseComp

  #14 / 68

  Finding hard-to-locate facts by browsing the web.

  84.3%

  Self-reported · In the agents score

• SWE-Bench Pro

  #5 / 61

  Harder, longer software engineering tasks from real codebases.

  69.2%

  Self-reported · In the coding score

• CharXiv-R

  #7 / 60

  Reading and reasoning about charts from scientific papers.

  89.9%

  Self-reported

• Terminal-Bench 2.0

  #7 / 54

  Getting real work done in a command-line terminal.

  74.6%

  Self-reported

• DeepSWE 1.1

  #35 / 50

  DeepSWE 1.1 evaluates software engineering agents using the mini-swe-agent harness.

  59.0%

  Self-reported · In the coding score

• SWE-bench Multilingual

  #4 / 49

  A multilingual benchmark for issue resolving in software engineering that covers Java, TypeScript, JavaScript, Go, Rust, C, and C++. Contains 1,632 high-quality instances carefully annotated from 2,456 candidates by 68 expert annotators, designed to evaluate Large Language Models across diverse software ecosystems beyond Python.

  84.4%

  Self-reported

• Toolathlon

  #11 / 42

  Tool Decathlon is a comprehensive benchmark for evaluating AI agents' ability to use multiple tools across diverse task categories. It measures proficiency in tool selection, sequencing, and execution across ten different tool-use scenarios.

  59.9%

  Self-reported · In the agents score

• LiveBench

  #6 / 38

  LiveBench is a challenging, contamination-limited LLM benchmark that addresses test set contamination by releasing new questions monthly based on recently-released datasets, arXiv papers, news articles, and IMDb movie synopses. It comprises tasks across math, coding, reasoning, language, instruction following, and data analysis with verifiable, objective ground-truth answers.

  77.2%

  Self-reported · In the text score

• MCP Atlas

  #7 / 38

  MCP Atlas is a benchmark for evaluating AI models on scaled tool use capabilities, measuring how well models can coordinate and utilize multiple tools across complex multi-step tasks.

  82.2%

  Self-reported

• Include

  #1 / 31

  Include benchmark - specific documentation not found in official sources

  87.6%

  Self-reported

• Finance Agent v2

  #6 / 28

  Finance Agent v2 is an agentic financial-analysis benchmark from Vals that evaluates models on real-world finance workflows, measuring their ability to retrieve and reason over financial documents, perform multi-step calculations, and produce accurate analyses.

  53.9%

  Self-reported

• OSWorld-Verified

  #4 / 28

  OSWorld-Verified is a verified subset of OSWorld, a scalable real computer environment for multimodal agents supporting task setup, execution-based evaluation, and interactive learning across Ubuntu, Windows, and macOS.

  83.4%

  Self-reported

• Terminal-Bench 4.0

  #21 / 28

  Terminal-Bench 4.0 evaluates AI agents on real-world work in terminal and command-line containerized environments across 66 tasks, with emphasis on science-adjacent and frontier engineering problems. Relative to earlier Terminal-Bench releases, 4.0 increases timeouts and adaptively raises RAM/CPU on selected tasks to reduce harness and resource confounds.

  23.6%

  Self-reported · In the coding score· In the agents score

• ScreenSpot Pro

  #3 / 27

  ScreenSpot-Pro is a novel GUI grounding benchmark designed to rigorously evaluate the grounding capabilities of multimodal large language models (MLLMs) in professional high-resolution computing environments. The benchmark comprises 1,581 instructions across 23 applications spanning 5 industries and 3 operating systems, featuring authentic high-resolution images from professional domains with expert annotations. Unlike previous benchmarks that focus on cropped screenshots in consumer applications, ScreenSpot-Pro addresses the complexity and diversity of real-world professional software scenarios, revealing significant performance gaps in current MLLM GUI perception capabilities.

  87.9%

  Self-reported

• FrontierCode 1.1

  #9 / 23

  FrontierCode 1.1 evaluates whether coding-agent changes are mergeable, using unit tests, maintainer-defined rubrics, and verifiers. Runs flagged for unfair internet use receive a zero score.

  46.5%

  Self-reported · In the coding score

• CyberGym

  #13 / 21

  CyberGym is a benchmark for evaluating AI agents on cybersecurity tasks, testing their ability to identify vulnerabilities, perform security analysis, and complete security-related challenges in a controlled environment.

  78.8%

  Self-reported

• HealthBench Professional

  #16 / 19

  HealthBench Professional evaluates model capability and safety for clinician use cases using real clinician-style chats and physician-authored grading rubrics.

  55.8%

  Self-reported

• FrontierSWE

  #4 / 16

  FrontierSWE measures whether an agent can complete open-ended technical projects at the scale of hours to tens of hours, spanning systems optimization, large-scale code construction, and applied ML research. Performance is reported as a dominance score, where higher is better.

  75.0%

  Self-reported

• Graphwalks BFS >128k

  #6 / 12

  A graph reasoning benchmark that evaluates language models' ability to perform breadth-first search (BFS) operations on graphs with context length over 128k tokens, testing long-context reasoning capabilities.

  68.1%

  Self-reported

• OfficeQA Pro

  #4 / 12

  OfficeQA Pro evaluates AI models on professional knowledge-work questions and tasks drawn from real office workflows, including document analysis, spreadsheet reasoning, and information synthesis across business domains.

  66.2%

  Self-reported

• DeepSearchQA

  #2 / 10

  DeepSearchQA is a benchmark for evaluating deep search and question-answering capabilities, testing models' ability to perform multi-hop reasoning and information retrieval across complex knowledge domains.

  93.1%

  Self-reported

• Finance Agent

  #7 / 8

  Finance Agent is a benchmark for evaluating AI models on agentic financial analysis tasks, testing their ability to process financial data, perform calculations, and generate accurate analyses across various financial domains.

  53.9%

  Self-reported

• Graphwalks parents >128k

  #2 / 7

  A graph reasoning benchmark that evaluates language models' ability to find parent nodes in graphs with context length over 128k tokens, testing long-context reasoning and graph structure understanding.

  83.3%

  Self-reported

• SWE-Bench Multimodal

  #5 / 7

  SWE-Bench Multimodal extends SWE-Bench to evaluate language models on software engineering tasks that involve visual inputs such as screenshots, UI mockups, and diagrams alongside code understanding.

  38.4%

  Self-reported