Gemma 4 31B

Leaderboards

Where Gemma 4 31B sits among the top 20 in each category.

By task

Scored from votes on each kind of prompt alone, on the same 1,000-point scale.

Test results

Every test Gemma 4 31B has a published result on, out of 100%, with its rank among the models that took it.

• GPQA

  #74 / 256

  Graduate-level science questions written to be hard to look up.

  84.3%

  Self-reported

• MMLU-Pro

  #19 / 142

  Broad knowledge and reasoning across 14 subjects, ten answer choices each.

  85.2%

  Self-reported

• Humanity's Last Exam

  #59 / 108

  Humanity's Last Exam (HLE) is a multi-modal academic benchmark with 2,500 questions across mathematics, humanities, and natural sciences, designed to test LLM capabilities at the frontier of human knowledge with unambiguous, verifiable solutions

  26.5%

  Self-reported · In the text score· In the vision score

• MMMU-Pro

  #25 / 73

  College-level questions that need both an image and text to answer.

  76.9%

  Self-reported · In the vision score

• LiveCodeBench v6

  #29 / 67

  LiveCodeBench is a holistic and contamination-free evaluation benchmark for large language models for code. It continuously collects new problems from programming contests (LeetCode, AtCoder, CodeForces) and evaluates four different scenarios: code generation, self-repair, code execution, and test output prediction. Problems are annotated with release dates to enable evaluation on unseen problems released after a model's training cutoff.

  80.0%

  Self-reported

• MMMLU

  #18 / 51

  Multilingual Massive Multitask Language Understanding dataset released by OpenAI, featuring professionally translated MMLU test questions across 14 languages including Arabic, Bengali, German, Spanish, French, Hindi, Indonesian, Italian, Japanese, Korean, Portuguese, Swahili, Yoruba, and Chinese. Contains approximately 15,908 multiple-choice questions per language covering 57 subjects.

  88.4%

  Self-reported

• Multi-Challenge

  #22 / 41

  MultiChallenge is a realistic multi-turn conversation evaluation benchmark that challenges frontier LLMs across four key categories: instruction retention (maintaining instructions throughout conversations), inference memory (recalling and connecting details from previous turns), reliable versioned editing (adapting to evolving instructions during collaborative editing), and self-coherence (avoiding contradictions in responses). The benchmark evaluates models on sustained, contextually complex dialogues across diverse topics including travel planning, technical documentation, and professional communication.

  51.5%

  Verified

• MathVision

  #12 / 38

  MATH-Vision is a dataset designed to measure multimodal mathematical reasoning capabilities. It focuses on evaluating how well models can solve mathematical problems that require both visual understanding and mathematical reasoning, bridging the gap between visual and mathematical domains.

  85.6%

  Self-reported

• MRCR v2 (8-needle)

  #8 / 32

  MRCR v2 (8-needle) is a variant of the Multi-Round Coreference Resolution benchmark that includes 8 needle items to retrieve from long contexts. This tests models' ability to simultaneously track and reason about multiple pieces of information across extended conversations.

  66.4%

  Verified · In the text score

• AIME 2026

  #19 / 26

  All 30 problems from the 2026 American Invitational Mathematics Examination (AIME I and AIME II), testing olympiad-level mathematical reasoning with integer answers from 000-999. Used as an AI benchmark to evaluate large language models' ability to solve complex mathematical problems requiring multi-step logical deductions and structured symbolic reasoning.

  89.2%

  Self-reported

• HealthBench Hard

  #18 / 24

  A challenging variation of HealthBench that evaluates large language models' performance and safety in healthcare through 5,000 multi-turn conversations with particularly rigorous evaluation criteria validated by 262 physicians from 60 countries

  20.1%

  Verified

• t2-bench

  #5 / 23

  t2-bench is a benchmark for evaluating agentic tool use capabilities, measuring how well models can select, sequence, and utilize tools to solve complex tasks. It tests autonomous planning and execution in multi-step scenarios.

  86.4%

  Self-reported

• FACTS Grounding

  #1 / 20

  A benchmark evaluating language models' ability to generate factually accurate and well-grounded responses based on long-form input context, comprising 1,719 examples with documents up to 32k tokens requiring detailed responses that are fully grounded in provided documents

  94.2%

  Verified

• HealthBench (hard)

  #10 / 13

  20.1%

  Unverified

• MedXpertQA

  #5 / 12

  A comprehensive benchmark to evaluate expert-level medical knowledge and advanced reasoning, featuring 4,460 questions spanning 17 specialties and 11 body systems. Includes both text-only and multimodal subsets with expert-level exam questions incorporating diverse medical images and rich clinical information.

  61.3%

  Self-reported

• BIG-Bench Extra Hard

  #1 / 11

  BIG-Bench Extra Hard (BBEH) is a challenging benchmark that replaces each task in BIG-Bench Hard with a novel task that probes similar reasoning capabilities but exhibits significantly increased difficulty. The benchmark contains 23 tasks testing diverse reasoning skills including many-hop reasoning, causal understanding, spatial reasoning, temporal arithmetic, geometric reasoning, linguistic reasoning, logic puzzles, and humor understanding. Designed to address saturation on existing benchmarks where state-of-the-art models achieve near-perfect scores, BBEH shows substantial room for improvement with best models achieving only 9.8-44.8% average accuracy.

  74.4%

  Self-reported

• mrcr-v2 (8needle_32768_65536)

  #4 / 10

  27.4%

  Unverified

• MultiChallenge (default)

  #3 / 9

  51.5%

  Unverified

• mrcr-v2 (8needle_upto_128k)

  #4 / 9

  33.2%

  Unverified

• facts-grounding (default)

  #1 / 8

  FACTS Grounding 1.0 — 860 public examples from Google DeepMind (CC-BY-4.0)

  94.2%

  Unverified