Qwen3.7-Plus

Leaderboards

Where Qwen3.7-Plus sits among the top 20 in each category.

By task

Scored from votes on each kind of prompt alone, on the same 1,000-point scale.

Test results

Every test Qwen3.7-Plus has a published result on, out of 100%, with its rank among the models that took it.

• GPQA

  #28 / 256

  Graduate-level science questions written to be hard to look up.

  90.3%

  Self-reported

• MMLU-Pro

  #3 / 142

  Broad knowledge and reasoning across 14 subjects, ten answer choices each.

  88.5%

  Self-reported

• SWE-Bench Verified

  #26 / 120

  Fixing real bugs in real GitHub projects, checked by the projects' own tests.

  77.7%

  Self-reported

• Humanity's Last Exam

  #49 / 108

  Humanity's Last Exam (HLE) is a multi-modal academic benchmark with 2,500 questions across mathematics, humanities, and natural sciences, designed to test LLM capabilities at the frontier of human knowledge with unambiguous, verifiable solutions

  34.7%

  Self-reported · In the text score· In the vision score

• MMMU-Pro

  #17 / 73

  College-level questions that need both an image and text to answer.

  79.0%

  Self-reported · In the vision score

• IFEval

  #2 / 70

  Instruction-Following Evaluation (IFEval) benchmark for large language models, focusing on verifiable instructions with 25 types of instructions and around 500 prompts containing one or more verifiable constraints

  94.6%

  Self-reported

• LiveCodeBench v6

  #6 / 67

  LiveCodeBench is a holistic and contamination-free evaluation benchmark for large language models for code. It continuously collects new problems from programming contests (LeetCode, AtCoder, CodeForces) and evaluates four different scenarios: code generation, self-repair, code execution, and test output prediction. Problems are annotated with release dates to enable evaluation on unseen problems released after a model's training cutoff.

  89.6%

  Self-reported

• SWE-Bench Pro

  #29 / 61

  Harder, longer software engineering tasks from real codebases.

  57.6%

  Self-reported · In the coding score

• CharXiv-R

  #17 / 60

  Reading and reasoning about charts from scientific papers.

  85.9%

  Self-reported

• Terminal-Bench 2.0

  #8 / 54

  Getting real work done in a command-line terminal.

  70.3%

  Self-reported

• MMMLU

  #14 / 51

  Multilingual Massive Multitask Language Understanding dataset released by OpenAI, featuring professionally translated MMLU test questions across 14 languages including Arabic, Bengali, German, Spanish, French, Hindi, Indonesian, Italian, Japanese, Korean, Portuguese, Swahili, Yoruba, and Chinese. Contains approximately 15,908 multiple-choice questions per language covering 57 subjects.

  89.0%

  Self-reported

• MMLU-Redux

  #3 / 50

  An improved version of the MMLU benchmark featuring manually re-annotated questions to identify and correct errors in the original dataset. Provides more reliable evaluation metrics for language models by addressing dataset quality issues found in the original MMLU.

  94.5%

  Self-reported

• SWE-bench Multilingual

  #16 / 49

  A multilingual benchmark for issue resolving in software engineering that covers Java, TypeScript, JavaScript, Go, Rust, C, and C++. Contains 1,632 high-quality instances carefully annotated from 2,456 candidates by 68 expert annotators, designed to evaluate Large Language Models across diverse software ecosystems beyond Python.

  75.8%

  Self-reported

• IFBench

  #11 / 44

  Instruction Following Benchmark evaluating model's ability to follow complex instructions

  79.1%

  Self-reported

• MCP Atlas

  #21 / 38

  MCP Atlas is a benchmark for evaluating AI models on scaled tool use capabilities, measuring how well models can coordinate and utilize multiple tools across complex multi-step tasks.

  73.2%

  Self-reported

• MathVision

  #8 / 38

  MATH-Vision is a dataset designed to measure multimodal mathematical reasoning capabilities. It focuses on evaluating how well models can solve mathematical problems that require both visual understanding and mathematical reasoning, bridging the gap between visual and mathematical domains.

  90.3%

  Self-reported

• SuperGPQA

  #3 / 37

  SuperGPQA is a comprehensive benchmark that evaluates large language models across 285 graduate-level academic disciplines. The benchmark contains 25,957 questions covering 13 broad disciplinary areas including Engineering, Medicine, Science, and Law, with specialized fields in light industry, agriculture, and service-oriented domains. It employs a Human-LLM collaborative filtering mechanism with over 80 expert annotators to create challenging questions that assess graduate-level knowledge and reasoning capabilities.

  71.4%

  Self-reported

• MMLU-ProX

  #2 / 32

  Extended version of MMLU-Pro providing additional challenging multiple-choice questions for evaluating language models across diverse academic and professional domains. Built on the foundation of the Massive Multitask Language Understanding benchmark framework.

  85.4%

  Self-reported

• RealWorldQA

  #4 / 32

  RealWorldQA is a benchmark designed to evaluate basic real-world spatial understanding capabilities of multimodal models. The initial release consists of over 700 anonymized images taken from vehicles and other real-world scenarios, each accompanied by a question and easily verifiable answer. Released by xAI as part of their Grok-1.5 Vision preview to test models' ability to understand natural scenes and spatial relationships in everyday visual contexts.

  86.9%

  Self-reported

• Include

  #5 / 31

  Include benchmark - specific documentation not found in official sources

  83.0%

  Self-reported

• LVBench

  #9 / 31

  LVBench is an extreme long video understanding benchmark designed to evaluate multimodal models on videos up to two hours in duration. It contains 6 major categories and 21 subcategories, with videos averaging five times longer than existing datasets. The benchmark addresses applications requiring comprehension of extremely long videos.

  76.2%

  Self-reported

• SciCode

  #8 / 30

  SciCode is a research coding benchmark curated by scientists that challenges language models to code solutions for scientific problems. It contains 338 subproblems decomposed from 80 challenging main problems across 16 natural science sub-fields including mathematics, physics, chemistry, biology, and materials science. Problems require knowledge recall, reasoning, and code synthesis skills.

  51.3%

  Self-reported

• ERQA

  #8 / 29

  Embodied Reasoning Question Answering benchmark consisting of 400 multiple-choice visual questions across spatial reasoning, trajectory reasoning, action reasoning, state estimation, and multi-view reasoning for evaluating AI capabilities in physical world interactions

  69.8%

  Self-reported

• Finance Agent v2

  #19 / 28

  Finance Agent v2 is an agentic financial-analysis benchmark from Vals that evaluates models on real-world finance workflows, measuring their ability to retrieve and reason over financial documents, perform multi-step calculations, and produce accurate analyses.

  38.2%

  Self-reported

• OSWorld-Verified

  #17 / 28

  OSWorld-Verified is a verified subset of OSWorld, a scalable real computer environment for multimodal agents supporting task setup, execution-based evaluation, and interactive learning across Ubuntu, Windows, and macOS.

  73.3%

  Self-reported

• VideoMMMU

  #5 / 28

  Video-MMMU evaluates Large Multimodal Models' ability to acquire knowledge from expert-level professional videos across six disciplines through three cognitive stages: perception, comprehension, and adaptation. Contains 300 videos and 900 human-annotated questions spanning Art, Business, Science, Medicine, Humanities, and Engineering.

  85.4%

  Self-reported

• ScreenSpot Pro

  #7 / 27

  ScreenSpot-Pro is a novel GUI grounding benchmark designed to rigorously evaluate the grounding capabilities of multimodal large language models (MLLMs) in professional high-resolution computing environments. The benchmark comprises 1,581 instructions across 23 applications spanning 5 industries and 3 operating systems, featuring authentic high-resolution images from professional domains with expert annotations. Unlike previous benchmarks that focus on cropped screenshots in consumer applications, ScreenSpot-Pro addresses the complexity and diversity of real-world professional software scenarios, revealing significant performance gaps in current MLLM GUI perception capabilities.

  79.0%

  Self-reported

• NL2Repo

  #21 / 25

  NL2Repo evaluates long-horizon coding capabilities including repository-level understanding, where models must generate or modify code across entire repositories from natural language specifications.

  41.1%

  Self-reported · In the coding score

• WMT24++

  #4 / 24

  WMT24++ is a comprehensive multilingual machine translation benchmark that expands the WMT24 dataset to cover 55 languages and dialects. It includes human-written references and post-edits across four domains (literary, news, social, and speech) to evaluate machine translation systems and large language models across diverse linguistic contexts.

  84.6%

  Self-reported

• FrontierCode 1.1

  #23 / 23

  FrontierCode 1.1 evaluates whether coding-agent changes are mergeable, using unit tests, maintainer-defined rubrics, and verifiers. Runs flagged for unfair internet use receive a zero score.

  10.2%

  Self-reported · In the coding score

• PolyMATH

  #2 / 23

  Polymath is a challenging multi-modal mathematical reasoning benchmark designed to evaluate the general cognitive reasoning abilities of Multi-modal Large Language Models (MLLMs). The benchmark comprises 5,000 manually collected high-quality images of cognitive textual and visual challenges across 10 distinct categories, including pattern recognition, spatial reasoning, and relative reasoning.

  84.0%

  Self-reported

• IMO-AnswerBench

  #7 / 22

  IMO-AnswerBench is a benchmark for evaluating mathematical reasoning capabilities on International Mathematical Olympiad (IMO) problems, focusing on answer generation and verification.

  86.0%

  Self-reported

• BFCL-V4

  #3 / 21

  Berkeley Function Calling Leaderboard V4 (BFCL-V4) evaluates LLMs on their ability to accurately call functions and APIs, including simple, multiple, parallel, and nested function calls across diverse programming scenarios.

  72.9%

  Self-reported

• OmniDocBench 1.5

  #2 / 20

  OmniDocBench 1.5 is a comprehensive benchmark for evaluating multimodal large language models on document understanding tasks, including OCR, document parsing, information extraction, and visual question answering across diverse document types. Lower Overall Edit Distance scores are better.

  91.4%

  Self-reported

• SimpleVQA

  #2 / 17

  SimpleVQA is a visual question answering benchmark focused on simple queries.

  0.8%

  Self-reported

• Video-MME

  #3 / 17

  Video-MME is the first-ever comprehensive evaluation benchmark of Multi-modal Large Language Models (MLLMs) in video analysis. It features 900 videos totaling 254 hours with 2,700 human-annotated question-answer pairs across 6 primary visual domains (Knowledge, Film & Television, Sports Competition, Life Record, Multilingual, and others) and 30 subfields. The benchmark evaluates models across diverse temporal dimensions (11 seconds to 1 hour), integrates multi-modal inputs including video frames, subtitles, and audio, and uses rigorous manual labeling by expert annotators for precise assessment.

  88.0%

  Self-reported

• BabyVision

  #9 / 16

  A benchmark for early-stage visual reasoning and perception on child-like vision tasks.

  70.4%

  Self-reported

• ODinW

  #2 / 16

  Object Detection in the Wild (ODinW) benchmark for evaluating object detection models' task-level transfer ability across diverse real-world datasets in terms of prediction accuracy and adaptation efficiency

  51.1%

  Self-reported

• Global PIQA

  #4 / 15

  Global PIQA is a multilingual commonsense reasoning benchmark that evaluates physical interaction knowledge across 100 languages and cultures. It tests AI systems' understanding of physical world knowledge in diverse cultural contexts through multiple choice questions about everyday situations requiring physical commonsense.

  90.3%

  Self-reported

• Claw-Eval

  #9 / 14

  Claw-Eval tests real-world agentic task completion across complex multi-step scenarios, evaluating a model's ability to use tools, navigate environments, and complete end-to-end tasks autonomously.

  62.7%

  Self-reported

• HMMT Feb 26

  #4 / 13

  HMMT February 2026 is a math competition benchmark based on problems from the Harvard-MIT Mathematics Tournament, testing advanced mathematical problem-solving and reasoning.

  92.9%

  Self-reported

• MAXIFE

  #2 / 12

  MAXIFE is a multilingual benchmark evaluating LLMs on instruction following and execution across multiple languages and cultural contexts.

  88.8%

  Self-reported

• NOVA-63

  #3 / 12

  NOVA-63 is a multilingual evaluation benchmark covering 63 languages, designed to assess LLM performance across diverse linguistic contexts and tasks.

  58.8%

  Self-reported

• VITA-Bench

  #3 / 11

  VITA-Bench evaluates AI agents on real-world virtual task automation, measuring their ability to complete complex multi-step tasks in simulated environments.

  45.6%

  Self-reported

• DeepPlanning

  #1 / 10

  DeepPlanning evaluates LLMs on complex multi-step planning tasks requiring long-horizon reasoning, goal decomposition, and strategic decision-making.

  62.3%

  Self-reported

• MLVU

  #1 / 10

  A comprehensive benchmark for multi-task long video understanding that evaluates multimodal large language models on videos ranging from 3 minutes to 2 hours across 9 distinct tasks including reasoning, captioning, recognition, and summarization.

  87.4%

  Self-reported

• SkillsBench

  #5 / 10

  SkillsBench evaluates coding agents on self-contained programming tasks, measuring practical engineering skills across diverse software development scenarios.

  54.9%

  Self-reported

• MCP-Mark

  #3 / 9

  MCP-Mark evaluates LLMs on their ability to use Model Context Protocol (MCP) tools effectively, testing tool discovery, selection, invocation, and result interpretation across diverse MCP server scenarios.

  58.7%

  Self-reported

• AndroidWorld

  #5 / 8

  AndroidWorld evaluates an agent's ability to operate in real Android GUI environments, completing multi-step tasks by perceiving screen content and executing touch/type actions.

  81.0%

  Self-reported

• OCRBench_V2

  #1 / 8

  OCRBench v2: Enhanced large-scale bilingual benchmark for evaluating Large Multimodal Models on visual text localization and reasoning with 10,000 human-verified question-answering pairs across 8 core OCR capabilities

  67.1%

  Self-reported

• WorldVQA

  #1 / 7

  WorldVQA is a benchmark designed to evaluate atomic vision-centric world knowledge. It assesses models' ability to understand and reason about visual elements representing real-world knowledge.

  61.1%

  Self-reported

• ClawEval-MM

  #4 / 6

  ClawEval-MM is the multimodal variant of ClawEval, evaluating agentic problem solving with visual inputs.

  55.7%

  Self-reported

• CoWorkBench

  #6 / 6

  CoWorkBench is Qwen's internal cowork benchmark for evaluating long-horizon office and productivity agent tasks across domains such as computer science, finance, law, and medicine.

  65.1%

  Self-reported

• CritPT

  #5 / 6

  CritPT is a challenging reasoning benchmark reported by Qwen for evaluating frontier mathematical and critical problem-solving capability.

  6.0%

  Self-reported

• TVBench

  #2 / 5

  TVBench is a temporal video understanding benchmark evaluating reasoning over actions, events, and temporal dynamics in videos.

  78.2%

  Self-reported

• LingoQA

  #1 / 4

  A benchmark for multimodal spatial-language understanding and visual-linguistic question answering.

  83.4%

  Self-reported

• SpreadSheetBench-v1

  #4 / 4

  SpreadSheetBench-v1 evaluates office automation agents on spreadsheet reasoning and manipulation tasks, measuring the ability to analyze, transform, and operate on spreadsheet data through tools.

  86.3%

  Self-reported

• VisFactor

  #3 / 4

  VisFactor is a benchmark evaluating fine-grained visual factor perception and reasoning over images.

  42.8%

  Self-reported

• MRCR v2

  #1 / 3

  MRCR v2 (Multi-Round Coreference Resolution version 2) is an enhanced version of the synthetic long-context reasoning task. It extends the original MRCR framework with improved evaluation criteria and additional complexity for testing models' ability to maintain attention and reasoning across extended contexts.

  91.7%

  Self-reported

• Apex

  #2 / 2

  Apex is a challenging frontier reasoning benchmark testing advanced multi-step problem solving across difficult STEM and logical tasks.

  22.7%

  Unverified

• HiPhO

  #1 / 2

  HiPhO is a high-school physics olympiad benchmark evaluating multimodal reasoning over physics problems that include diagrams and figures.

  84.1%

  Unverified

• MMSearch-Plus

  #1 / 2

  MMSearch-Plus is an extended variant of MMSearch with harder multimodal search and retrieval tasks requiring deeper reasoning over visual and textual search results.

  41.4%

  Unverified

• QwenWorldBench

  #1 / 2

  QwenWorldBench is Qwen's internal benchmark for evaluating LLMs as world models that simulate agentic environments across Terminal, SWE, MCP, Search, OS, Android, and Web domains.

  62.1%

  Unverified

• BC-VL

  #1 / 1

  BC-VL is a vision-language benchmark for knowledge-grounded multimodal question answering.

  51.1%

  Unverified

• CountQA

  #1 / 1

  CountQA is a benchmark for visual object counting and quantity reasoning over images.

  77.0%

  Unverified

• MMBC

  #1 / 1

  MMBC is a multimodal benchmark for vision-language knowledge and reasoning.

  46.3%

  Unverified

• MedXpertQA-MM

  #1 / 1

  MedXpertQA-MM is the multimodal subset of MedXpertQA, evaluating expert-level medical question answering grounded in medical images.

  71.0%

  Unverified

• QwenClawBench

  #1 / 1

  QwenClawBench is a real-user-distribution Claw agent benchmark for evaluating coding agents on realistic developer tasks.

  61.8%

  Unverified

• SURDS

  #1 / 1

  SURDS is a benchmark for spatial understanding and reasoning in autonomous-driving scenes.

  77.2%

  Unverified

• VLADBench

  #1 / 1

  VLADBench is a vision-language autonomous-driving benchmark evaluating understanding of dynamic traffic scenes and participants.

  77.2%

  Unverified