Skip to content

NVIDIA llama_3_1_community_license Released Apr 7, 2025 Updated

Llama 3.1 Nemotron Ultra 253B v1

ToolsGuide Scores, rankings, test results and price.

Model summary

  • Text #211 / 416

    230 / 1,000

    ToolsGuide Score· votes 219, tests 243

  • Price per 1M tokens

    No listed price

Summary

Llama 3.1 Nemotron Ultra 253B v1 is NVIDIA's model, released Apr 7, 2025. It's ranked in 1 category and does best at text: #211 of 416, with a ToolsGuide Score of 230 against a median of 235.

It has published results on 6 tests.

Specifications

Creator
NVIDIA
Licence
llama_3_1_community_license
Released
Apr 7, 2025
Context window
–
Input price
–
Output price
–
Tests reported
6
Settings ranked
llama-3.1-nemotron-ultra-253b-v1

Model comparison

Against the ten best text models.

Text score

ToolsGuide Score out of 1,000 · Higher is better

Input price

USD per 1M input tokens · Lower is better

Leaderboards

Where Llama 3.1 Nemotron Ultra 253B v1 sits among the top 20 in each category.

Text

#211 of 416 · ToolsGuide Score out of 1,000, where 1,000 is a flawless model

Full text leaderboard →

By task

Scored from votes on each kind of prompt alone, on the same 1,000-point scale.

Text

llama-3.1-nemotron-ultra-253b-v1 · hover a bar for its rank

Test results

Every test Llama 3.1 Nemotron Ultra 253B v1 has a published result on, out of 100%, with its rank among the models that took it.

  • GPQA

    #118 / 256

    Graduate-level science questions written to be hard to look up.

    76.0%

    Self-reported

  • AIME 2025

    #90 / 123

    Competition maths problems from the 2025 American Invitational Mathematics Exam.

    72.5%

    Self-reported

  • LiveCodeBench

    #28 / 76

    LiveCodeBench is a holistic and contamination-free evaluation benchmark for large language models for code. It continuously collects new problems from programming contests (LeetCode, AtCoder, CodeForces) and evaluates four different scenarios: code generation, self-repair, code execution, and test output prediction. Problems are annotated with release dates to enable evaluation on unseen problems released after a model's training cutoff.

    66.3%

    Self-reported

  • IFEval

    #23 / 70

    Instruction-Following Evaluation (IFEval) benchmark for large language models, focusing on verifiable instructions with 25 types of instructions and around 500 prompts containing one or more verifiable constraints

    89.5%

    Self-reported

  • MATH-500

    #8 / 34

    MATH-500 is a subset of the MATH dataset containing 500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem includes full step-by-step solutions and spans multiple difficulty levels across seven mathematical subjects including Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Precalculus.

    97.0%

    Self-reported

  • BFCL v2

    #2 / 5

    Berkeley Function Calling Leaderboard (BFCL) v2 is a comprehensive benchmark for evaluating large language models' function calling capabilities. It features 2,251 question-function-answer pairs with enterprise and OSS-contributed functions, addressing data contamination and bias through live, user-contributed scenarios. The benchmark evaluates AST accuracy, executable accuracy, irrelevance detection, and relevance detection across multiple programming languages (Python, Java, JavaScript) and includes complex real-world function calling scenarios with multi-lingual prompts.

    74.1%

    Self-reported