Qwen3.8 Max

Leaderboards

Where Qwen3.8 Max sits among the top 20 in each category.

By task

Scored from votes on each kind of prompt alone, on the same 1,000-point scale.

Test results

Every test Qwen3.8 Max has a published result on, out of 100%, with its rank among the models that took it.

• GPQA

  #13 / 256

  Graduate-level science questions written to be hard to look up.

  92.6%

  Self-reported

• Humanity's Last Exam

  #32 / 108

  Humanity's Last Exam (HLE) is a multi-modal academic benchmark with 2,500 questions across mathematics, humanities, and natural sciences, designed to test LLM capabilities at the frontier of human knowledge with unambiguous, verifiable solutions

  43.6%

  Self-reported · In the text score· In the vision score

• MMMU-Pro

  #6 / 73

  College-level questions that need both an image and text to answer.

  82.3%

  Self-reported · In the vision score

• SWE-Bench Pro

  #6 / 61

  Harder, longer software engineering tasks from real codebases.

  67.7%

  Self-reported · In the coding score

• DeepSWE 1.1

  #38 / 50

  DeepSWE 1.1 evaluates software engineering agents using the mini-swe-agent harness.

  56.6%

  Self-reported · In the coding score

• Terminal-Bench 2.1

  #11 / 46

  Terminal-Bench 2.1 is an updated release of the Terminal-Bench benchmark that tests AI agents' ability to operate a computer via the terminal. It evaluates how well models handle real-world, end-to-end tasks autonomously, including compiling code, training models, setting up servers, system administration, data science workflows, and security tasks.

  86.6%

  Self-reported

• IFBench

  #1 / 44

  Instruction Following Benchmark evaluating model's ability to follow complex instructions

  82.8%

  Self-reported

• Toolathlon

  #9 / 42

  Tool Decathlon is a comprehensive benchmark for evaluating AI agents' ability to use multiple tools across diverse task categories. It measures proficiency in tool selection, sequencing, and execution across ten different tool-use scenarios.

  72.5%

  Self-reported · In the agents score

• MRCR v2 (8-needle)

  #3 / 32

  MRCR v2 (8-needle) is a variant of the Multi-Round Coreference Resolution benchmark that includes 8 needle items to retrieve from long contexts. This tests models' ability to simultaneously track and reason about multiple pieces of information across extended conversations.

  92.9%

  Self-reported · In the text score

• RealWorldQA

  #3 / 32

  RealWorldQA is a benchmark designed to evaluate basic real-world spatial understanding capabilities of multimodal models. The initial release consists of over 700 anonymized images taken from vehicles and other real-world scenarios, each accompanied by a question and easily verifiable answer. Released by xAI as part of their Grok-1.5 Vision preview to test models' ability to understand natural scenes and spatial relationships in everyday visual contexts.

  88.0%

  Self-reported

• LVBench

  #4 / 31

  LVBench is an extreme long video understanding benchmark designed to evaluate multimodal models on videos up to two hours in duration. It contains 6 major categories and 21 subcategories, with videos averaging five times longer than existing datasets. The benchmark addresses applications requiring comprehension of extremely long videos.

  81.8%

  Self-reported

• ERQA

  #3 / 29

  Embodied Reasoning Question Answering benchmark consisting of 400 multiple-choice visual questions across spatial reasoning, trajectory reasoning, action reasoning, state estimation, and multi-view reasoning for evaluating AI capabilities in physical world interactions

  77.8%

  Self-reported

• OSWorld-Verified

  #1 / 28

  OSWorld-Verified is a verified subset of OSWorld, a scalable real computer environment for multimodal agents supporting task setup, execution-based evaluation, and interactive learning across Ubuntu, Windows, and macOS.

  86.1%

  Self-reported

• ScreenSpot Pro

  #5 / 27

  ScreenSpot-Pro is a novel GUI grounding benchmark designed to rigorously evaluate the grounding capabilities of multimodal large language models (MLLMs) in professional high-resolution computing environments. The benchmark comprises 1,581 instructions across 23 applications spanning 5 industries and 3 operating systems, featuring authentic high-resolution images from professional domains with expert annotations. Unlike previous benchmarks that focus on cropped screenshots in consumer applications, ScreenSpot-Pro addresses the complexity and diversity of real-world professional software scenarios, revealing significant performance gaps in current MLLM GUI perception capabilities.

  84.5%

  Self-reported

• NL2Repo

  #9 / 25

  NL2Repo evaluates long-horizon coding capabilities including repository-level understanding, where models must generate or modify code across entire repositories from natural language specifications.

  55.9%

  Self-reported · In the coding score

• Agents' Last Exam

  #4 / 24

  Agents' Last Exam is a challenging benchmark for AI agents on hard, long-horizon tasks that test sustained reasoning, planning, and tool use, reported with and without tool access.

  52.4%

  Self-reported · In the agents score

• AutomationBench

  #16 / 24

  AutomationBench is a tool-use benchmark that evaluates AI agents on automating real-world workflows, testing their ability to orchestrate tools and complete multi-step automation tasks.

  27.3%

  Self-reported · In the agents score

• LongBench v2

  #1 / 24

  LongBench v2 is a benchmark designed to assess the ability of LLMs to handle long-context problems requiring deep understanding and reasoning across real-world multitasks. It consists of 503 challenging multiple-choice questions with contexts ranging from 8k to 2M words across six major task categories: single-document QA, multi-document QA, long in-context learning, long-dialogue history understanding, code repository understanding, and long structured data understanding.

  66.3%

  Self-reported

• FrontierSWE

  #6 / 16

  FrontierSWE measures whether an agent can complete open-ended technical projects at the scale of hours to tens of hours, spanning systems optimization, large-scale code construction, and applied ML research. Performance is reported as a dominance score, where higher is better.

  73.5%

  Self-reported

• HealthBench

  #4 / 14

  An open-source benchmark for measuring performance and safety of large language models in healthcare, consisting of 5,000 multi-turn conversations evaluated by 262 physicians using 48,562 unique rubric criteria across health contexts and behavioral dimensions

  60.2%

  Self-reported

• WideSearch

  #2 / 14

  WideSearch is an agentic search benchmark that evaluates models' ability to perform broad, parallel search operations across multiple sources. It tests wide-coverage information retrieval and synthesis capabilities.

  81.9%

  Self-reported

• VideoMME w sub.

  #1 / 12

  The first-ever comprehensive evaluation benchmark of Multi-modal LLMs in Video analysis. Features 900 videos (254 hours) with 2,700 question-answer pairs covering 6 primary visual domains and 30 subfields. Evaluates temporal understanding across short (11 seconds) to long (1 hour) videos with multi-modal inputs including video frames, subtitles, and audio.

  90.4%

  Self-reported

• Humanity's Last Exam (with tools, text-only)

  #4 / 10

  Text-only Humanity's Last Exam variant evaluated with tool use enabled.

  56.2%

  Self-reported

• SkillsBench

  #1 / 10

  SkillsBench evaluates coding agents on self-contained programming tasks, measuring practical engineering skills across diverse software development scenarios.

  70.2%

  Self-reported

• Job Bench

  #7 / 9

  Job Bench evaluates AI agents on realistic professional tasks that require multi-step planning, research, and production of work artifacts.

  53.4%

  Self-reported

• AndroidWorld

  #1 / 8

  AndroidWorld evaluates an agent's ability to operate in real Android GUI environments, completing multi-step tasks by perceiving screen content and executing touch/type actions.

  85.3%

  Self-reported

• CoWorkBench

  #1 / 6

  CoWorkBench is Qwen's internal cowork benchmark for evaluating long-horizon office and productivity agent tasks across domains such as computer science, finance, law, and medicine.

  74.8%

  Self-reported

• Vision2Web

  #1 / 5

  Vision2Web evaluates multimodal models on converting visual designs and screenshots into functional web pages, measuring end-to-end design-to-code capability.

  69.0%

  Self-reported

• OneMillion Bench

  #4 / 4

  OneMillion Bench evaluates AI agents on high-economic-value tasks that require sustained, reliable execution across long-horizon real-world workflows.

  52.5%

  Self-reported

• Workspace Bench

  #1 / 4

  Workspace Bench evaluates AI agents on high-economic-value workplace tasks that span multi-step planning, file processing, and tool use across realistic office and productivity workflows.

  67.7%

  Self-reported

• MLS-Bench Lite

  #2 / 3

  MLS-Bench Lite is the official 30-task subset of MLS-Bench for evaluating whether AI systems can invent generalizable and scalable machine learning methods across LLM pretraining and post-training, robotics, world models, computer vision, reinforcement learning, optimization, ML systems, and AI for Science.

  41.0%

  Self-reported

• MobileWorld

  #1 / 3

  MobileWorld is a benchmark for evaluating multimodal agents on real mobile-device tasks, testing GUI grounding, navigation, and multi-step task completion in mobile environments.

  77.8%

  Self-reported

• PaperBench

  #1 / 3

  PaperBench is a benchmark for evaluating AI agents on their ability to replicate research papers. It tests models on complex, multi-step workflows involving code implementation, experimentation, and reproducing scientific results from academic publications.

  93.0%

  Self-reported

• PerceptionBench

  #1 / 2

  PerceptionBench is Moonshot AI's internal benchmark for evaluating atomic visual perception capabilities.

  63.5%

  Unverified

• QwenSVG

  #1 / 2

  QwenSVG is Qwen's internal SVG generation benchmark for evaluating front-end and visual code generation. Scores are reported as BT/Elo ratings from auto-rendered outputs judged by a multimodal evaluator.

  85.7%

  Unverified

• QwenSWEBench

  #1 / 2

  QwenSWEBench is Qwen's software-engineering agent benchmark for repository-level issue resolution.

  80.7%

  Unverified

• AndroidBench

  #1 / 1

  AndroidBench evaluates coding agents on Android application development tasks.

  75.1%

  Unverified

• PLawBench

  #1 / 1

  PLawBench evaluates language models on professional legal knowledge and reasoning tasks.

  73.2%

  Unverified

• PRBench-Finance

  #1 / 1

  PRBench-Finance evaluates professional reasoning on finance tasks.

  58.3%

  Unverified

• PRBench-Legal

  #1 / 1

  PRBench-Legal evaluates professional reasoning on legal tasks.

  57.6%

  Unverified

• QwenQoderBench

  #1 / 1

  QwenQoderBench is Qwen's internal benchmark for evaluating coding-agent performance.

  58.4%

  Unverified

• QwenReactBench

  #1 / 1

  QwenReactBench is Qwen's internal React application generation benchmark, reported as a BT/Elo rating.

  86.2%

  Unverified