Mistral Small 3 24B Instruct

Leaderboards

Where Mistral Small 3 24B Instruct sits among the top 20 in each category.

By task

Scored from votes on each kind of prompt alone, on the same 1,000-point scale.

Test results

Every test Mistral Small 3 24B Instruct has a published result on, out of 100%, with its rank among the models that took it.

• GPQA

  #214 / 256

  Graduate-level science questions written to be hard to look up.

  45.3%

  Self-reported

• MMLU-Pro

  #110 / 142

  Broad knowledge and reasoning across 14 subjects, ten answer choices each.

  66.3%

  Self-reported

• MATH

  #37 / 71

  MATH dataset contains 12,500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem includes full step-by-step solutions and spans multiple difficulty levels (1-5) across seven mathematical subjects including Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Precalculus.

  70.6%

  Self-reported

• IFEval

  #50 / 70

  Instruction-Following Evaluation (IFEval) benchmark for large language models, focusing on verifiable instructions with 25 types of instructions and around 500 prompts containing one or more verifiable constraints

  82.9%

  Self-reported

• HumanEval

  #37 / 66

  A benchmark that measures functional correctness for synthesizing programs from docstrings, consisting of 164 original programming problems assessing language comprehension, algorithms, and simple mathematics

  84.8%

  Self-reported

• Arena Hard

  #6 / 27

  Arena-Hard-Auto is an automatic evaluation benchmark for instruction-tuned LLMs consisting of 500 challenging real-world prompts curated by BenchBuilder. It includes open-ended software engineering problems, mathematical questions, and creative writing tasks. The benchmark uses LLM-as-a-Judge methodology with GPT-4.1 and Gemini-2.5 as automatic judges to approximate human preference. Arena-Hard achieves 98.6% correlation with human preference rankings and provides 3x higher separation of model performances compared to MT-Bench, making it highly effective for distinguishing between models of similar quality.

  87.6%

  Self-reported

• MT-Bench

  #9 / 13

  MT-Bench is a challenging multi-turn benchmark that measures the ability of large language models to engage in coherent, informative, and engaging conversations. It uses strong LLMs as judges for scalable and explainable evaluation of multi-turn dialogue capabilities.

  0.8%

  Self-reported

• Wild Bench

  #6 / 8

  WildBench is an automated evaluation framework that benchmarks large language models using 1,024 challenging, real-world tasks selected from over one million human-chatbot conversation logs. It introduces two evaluation metrics (WB-Reward and WB-Score) that achieve high correlation with human preferences and uses task-specific checklists for systematic evaluation.

  52.2%

  Self-reported