Claude Haiku 4.5

Leaderboards

Where Claude Haiku 4.5 sits among the top 20 in each category.

By task

Scored from votes on each kind of prompt alone, on the same 1,000-point scale.

Test results

Every test Claude Haiku 4.5 has a published result on, out of 100%, with its rank among the models that took it.

• GPQA

  #138 / 256

  Graduate-level science questions written to be hard to look up.

  73.0%

  Self-reported

• AIME 2025

  #75 / 123

  Competition maths problems from the 2025 American Invitational Mathematics Exam.

  80.7%

  Self-reported

• SWE-Bench Verified

  #51 / 120

  Fixing real bugs in real GitHub projects, checked by the projects' own tests.

  73.3%

  Self-reported

• MMMLU

  #37 / 51

  Multilingual Massive Multitask Language Understanding dataset released by OpenAI, featuring professionally translated MMLU test questions across 14 languages including Arabic, Bengali, German, Spanish, French, Hindi, Indonesian, Italian, Japanese, Korean, Portuguese, Swahili, Yoruba, and Chinese. Contains approximately 15,908 multiple-choice questions per language covering 57 subjects.

  83.0%

  Self-reported

• Multi-Challenge

  #35 / 41

  MultiChallenge is a realistic multi-turn conversation evaluation benchmark that challenges frontier LLMs across four key categories: instruction retention (maintaining instructions throughout conversations), inference memory (recalling and connecting details from previous turns), reliable versioned editing (adapting to evolving instructions during collaborative editing), and self-coherence (avoiding contradictions in responses). The benchmark evaluates models on sustained, contextually complex dialogues across diverse topics including travel planning, technical documentation, and professional communication.

  36.5%

  Verified

• Tau2 Telecom

  #21 / 36

  τ²-Bench telecom domain evaluates conversational agents in a dual-control environment modeled as a Dec-POMDP, where both agent and user use tools in shared telecommunications troubleshooting scenarios that test coordination and communication capabilities.

  83.0%

  Self-reported

• MRCR v2 (8-needle)

  #30 / 32

  MRCR v2 (8-needle) is a variant of the Multi-Round Coreference Resolution benchmark that includes 8 needle items to retrieve from long contexts. This tests models' ability to simultaneously track and reason about multiple pieces of information across extended conversations.

  15.0%

  Verified · In the text score

• Finance Agent v2

  #25 / 28

  Finance Agent v2 is an agentic financial-analysis benchmark from Vals that evaluates models on real-world finance workflows, measuring their ability to retrieve and reason over financial documents, perform multi-step calculations, and produce accurate analyses.

  31.0%

  Self-reported

• Tau2 Retail

  #5 / 27

  τ²-bench retail domain evaluates conversational AI agents in customer service scenarios within a dual-control environment where both agent and user can interact with tools. Tests tool-agent-user interaction, rule adherence, and task consistency in retail customer support contexts.

  83.2%

  Self-reported

• Tau2 Airline

  #11 / 25

  TAU2 airline domain benchmark for evaluating conversational agents in dual-control environments where both AI agents and users interact with tools in airline customer service scenarios. Tests agent coordination, communication, and ability to guide user actions in tasks like flight booking, modifications, cancellations, and refunds.

  63.6%

  Self-reported

• Terminal-Bench

  #7 / 25

  Terminal-Bench is a benchmark for testing AI agents in real terminal environments. It evaluates how well agents can handle real-world, end-to-end tasks autonomously, including compiling code, training models, setting up servers, system administration, security tasks, data science workflows, and cybersecurity vulnerabilities. The benchmark consists of a dataset of ~100 hand-crafted, human-verified tasks and an execution harness that connects language models to a terminal sandbox.

  41.0%

  Self-reported

• HealthBench Hard

  #20 / 24

  A challenging variation of HealthBench that evaluates large language models' performance and safety in healthcare through 5,000 multi-turn conversations with particularly rigorous evaluation criteria validated by 262 physicians from 60 countries

  17.3%

  Verified

• OSWorld

  #10 / 21

  OSWorld: The first-of-its-kind scalable, real computer environment for multimodal agents, supporting task setup, execution-based evaluation, and interactive learning across Ubuntu, Windows, and macOS with 369 computer tasks involving real web and desktop applications, OS file I/O, and multi-application workflows

  50.7%

  Self-reported

• FACTS Grounding

  #14 / 20

  A benchmark evaluating language models' ability to generate factually accurate and well-grounded responses based on long-form input context, comprising 1,719 examples with documents up to 32k tokens requiring detailed responses that are fully grounded in provided documents

  63.8%

  Verified

• HealthBench (hard)

  #13 / 13

  17.3%

  Unverified

• mrcr-v2 (8needle_32768_65536)

  #9 / 10

  15.0%

  Unverified

• mrcr-v2 (8needle_4096_8192)

  #9 / 10

  26.3%

  Unverified

• MultiChallenge (default)

  #7 / 9

  36.5%

  Unverified

• mrcr-v2 (8needle_upto_128k)

  #8 / 9

  16.6%

  Unverified

• facts-grounding (default)

  #5 / 8

  FACTS Grounding 1.0 — 860 public examples from Google DeepMind (CC-BY-4.0)

  63.8%

  Unverified

• nolima (hard_16k)

  #6 / 6

  7.2%

  Independent

• nolima (hard_4k)

  #5 / 5

  14.8%

  Independent

• MMMU (validation)

  #4 / 4

  Validation set of the Massive Multi-discipline Multimodal Understanding and Reasoning benchmark. Features college-level multimodal questions across 6 core disciplines (Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, Tech & Engineering) spanning 30 subjects and 183 subfields with diverse image types including charts, diagrams, maps, and tables.

  73.2%

  Self-reported

• Cybersecurity CTFs

  #2 / 3

  Cybersecurity Capture the Flag (CTF) benchmark for evaluating LLMs in offensive security challenges. Contains diverse cybersecurity tasks including cryptography, web exploitation, binary analysis, and forensics to assess AI capabilities in cybersecurity problem-solving.

  46.9%

  Self-reported