Llama-3.3 Nemotron Super 49B v1

Leaderboards

Where Llama-3.3 Nemotron Super 49B v1 sits among the top 20 in each category.

By task

Scored from votes on each kind of prompt alone, on the same 1,000-point scale.

Test results

Every test Llama-3.3 Nemotron Super 49B v1 has a published result on, out of 100%, with its rank among the models that took it.

• GPQA

  #169 / 265

  Graduate-level science questions written to be hard to look up.

  66.7%

  Self-reported

• AIME 2025

  #106 / 124

  Competition maths problems from the 2025 American Invitational Mathematics Exam.

  58.4%

  Self-reported

• MATH-500

  #13 / 34

  MATH-500 is a subset of the MATH dataset containing 500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem includes full step-by-step solutions and spans multiple difficulty levels across seven mathematical subjects including Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Precalculus.

  96.6%

  Self-reported

• MBPP

  #2 / 33

  MBPP (Mostly Basic Python Problems) is a benchmark of 974 crowd-sourced Python programming problems designed to be solvable by entry-level programmers. Each problem consists of a task description, code solution, and 3 automated test cases covering programming fundamentals and standard library functionality.

  0.9%

  Self-reported

• Arena Hard

  #5 / 27

  Arena-Hard-Auto is an automatic evaluation benchmark for instruction-tuned LLMs consisting of 500 challenging real-world prompts curated by BenchBuilder. It includes open-ended software engineering problems, mathematical questions, and creative writing tasks. The benchmark uses LLM-as-a-Judge methodology with GPT-4.1 and Gemini-2.5 as automatic judges to approximate human preference. Arena-Hard achieves 98.6% correlation with human preference rankings and provides 3x higher separation of model performances compared to MT-Bench, making it highly effective for distinguishing between models of similar quality.

  88.3%

  Self-reported

• MT-Bench

  #4 / 13

  MT-Bench is a challenging multi-turn benchmark that measures the ability of large language models to engage in coherent, informative, and engaging conversations. It uses strong LLMs as judges for scalable and explainable evaluation of multi-turn dialogue capabilities.

  0.9%

  Self-reported

• BFCL v2

  #3 / 5

  Berkeley Function Calling Leaderboard (BFCL) v2 is a comprehensive benchmark for evaluating large language models' function calling capabilities. It features 2,251 question-function-answer pairs with enterprise and OSS-contributed functions, addressing data contamination and bias through live, user-contributed scenarios. The benchmark evaluates AST accuracy, executable accuracy, irrelevance detection, and relevance detection across multiple programming languages (Python, Java, JavaScript) and includes complex real-world function calling scenarios with multi-lingual prompts.

  73.7%

  Self-reported