Skip to content

Anthropic Proprietary Released May 22, 2025 Updated

Claude Sonnet 4

ToolsGuide Scores, rankings, test results and price.

Model summary

  • Text #181 / 416

    263 / 1,000

    ToolsGuide Score· votes 278, tests 249

  • Vision #86 / 167

    338 / 1,000

    ToolsGuide Score· votes only

  • Price per 1M tokens

    No listed price

Summary

Claude Sonnet 4 is Anthropic's model, released May 22, 2025. It's ranked in 2 categories and does best at text: #181 of 416, with a ToolsGuide Score of 263 against a median of 235.

It also ranks #86 of 167 for vision (338).

Its vision score uses votes only, because it has published fewer than two of the tests used for that category.

It has published results on 8 tests.

Specifications

Creator
Anthropic
Licence
Proprietary
Released
May 22, 2025
Context window
–
Input price
–
Output price
–
Tests reported
8
Settings ranked
claude-sonnet-4-20250514, Claude Sonnet 4 (Thinking 32K, 20250514), Claude Sonnet 4 (20250514)

Model comparison

Against the ten best text models.

Text score

ToolsGuide Score out of 1,000 · Higher is better

Vision score

ToolsGuide Score out of 1,000 · Higher is better

Input price

USD per 1M input tokens · Lower is better

Leaderboards

Where Claude Sonnet 4 sits among the top 20 in each category.

Text

#181 of 416 · ToolsGuide Score out of 1,000, where 1,000 is a flawless model

Full text leaderboard →

Vision

#86 of 167 · ToolsGuide Score out of 1,000, where 1,000 is a flawless model

Full vision leaderboard →

By task

Scored from votes on each kind of prompt alone, on the same 1,000-point scale.

Against its nearest rivals

Text prompts by kind · ToolsGuide Score from votes · hover for values

Text

claude-sonnet-4-20250514 · hover a bar for its rank

Vision

Claude Sonnet 4 (Thinking 32K, 20250514) · hover a bar for its rank

Test results

Every test Claude Sonnet 4 has a published result on, out of 100%, with its rank among the models that took it.

  • GPQA

    #121 / 256

    Graduate-level science questions written to be hard to look up.

    75.4%

    Self-reported

  • AIME 2025

    #95 / 123

    Competition maths problems from the 2025 American Invitational Mathematics Exam.

    70.5%

    Self-reported

  • SWE-Bench Verified

    #56 / 120

    Fixing real bugs in real GitHub projects, checked by the projects' own tests.

    72.7%

    Self-reported

  • MMMU

    #27 / 66

    MMMU (Massive Multi-discipline Multimodal Understanding) is a benchmark designed to evaluate multimodal models on college-level subject knowledge and deliberate reasoning. Contains 11.5K meticulously collected multimodal questions from college exams, quizzes, and textbooks, covering six core disciplines: Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering across 30 subjects and 183 subfields.

    74.4%

    Self-reported

  • MMMLU

    #23 / 51

    Multilingual Massive Multitask Language Understanding dataset released by OpenAI, featuring professionally translated MMLU test questions across 14 languages including Arabic, Bengali, German, Spanish, French, Hindi, Indonesian, Italian, Japanese, Korean, Portuguese, Swahili, Yoruba, and Chinese. Contains approximately 15,908 multiple-choice questions per language covering 57 subjects.

    86.5%

    Self-reported

  • TAU-bench Retail

    #5 / 25

    A benchmark for evaluating tool-agent-user interaction in retail environments. Tests language agents' ability to handle dynamic conversations with users while using domain-specific API tools and following policy guidelines. Evaluates agents on tasks like order cancellations, address changes, and order status checks through multi-turn conversations.

    80.5%

    Self-reported

  • Terminal-Bench

    #13 / 25

    Terminal-Bench is a benchmark for testing AI agents in real terminal environments. It evaluates how well agents can handle real-world, end-to-end tasks autonomously, including compiling code, training models, setting up servers, system administration, security tasks, data science workflows, and cybersecurity vulnerabilities. The benchmark consists of a dataset of ~100 hand-crafted, human-verified tasks and an execution harness that connects language models to a terminal sandbox.

    35.5%

    Self-reported

  • TAU-bench Airline

    #5 / 23

    Part of τ-bench (TAU-bench), a benchmark for Tool-Agent-User interaction in real-world domains. The airline domain evaluates language agents' ability to interact with users through dynamic conversations while following domain-specific rules and using API tools. Agents must handle airline-related tasks and policies reliably.

    60.0%

    Self-reported