Skip to content

Meta llama_3_3_community_license_agreement Released Dec 6, 2024 Updated

Llama 3.3 70B Instruct

ToolsGuide Scores, rankings, test results and price.

Model summary

  • Text #246 / 416

    196 / 1,000

    ToolsGuide Score· votes 192, tests 200

  • Price per 1M tokens

    In $0.10· Out $0.32

    $0.10

    Input · median text model $0.43

Summary

Llama 3.3 70B Instruct is Meta's model, released Dec 6, 2024. It's ranked in 1 category and does best at text: #246 of 416, with a ToolsGuide Score of 196 against a median of 235.

It costs $0.10 per million input tokens and $0.32 per million output tokens, below the median text model ($0.43).

It has published results on 9 tests and has the top score on 1 of them, including BFCL v2.

Specifications

Creator
Meta
Licence
llama_3_3_community_license_agreement
Released
Dec 6, 2024
Context window
131K tokens
Input price
$0.10 / 1M
Output price
$0.32 / 1M
Tests reported
9
Settings ranked
llama-3.3-70b-instruct

Model comparison

Against the ten best text models.

Text score

ToolsGuide Score out of 1,000 · Higher is better

Input price

USD per 1M input tokens · Lower is better

Leaderboards

Where Llama 3.3 70B Instruct sits among the top 20 in each category.

Text

#246 of 416 · ToolsGuide Score out of 1,000, where 1,000 is a flawless model

Full text leaderboard →

By task

Scored from votes on each kind of prompt alone, on the same 1,000-point scale.

Score against price

Text · the 10 priced models ranked closest · up and to the left is better value

Against its nearest rivals

Text prompts by kind · ToolsGuide Score from votes · hover for values

Text

llama-3.3-70b-instruct · hover a bar for its rank

Test results

Every test Llama 3.3 70B Instruct has a published result on, out of 100%, with its rank among the models that took it.

  • GPQA

    #198 / 256

    Graduate-level science questions written to be hard to look up.

    50.5%

    Self-reported

  • MMLU-Pro

    #97 / 142

    Broad knowledge and reasoning across 14 subjects, ten answer choices each.

    68.9%

    Self-reported

  • MMLU

    #34 / 103

    Massive Multitask Language Understanding benchmark testing knowledge across 57 diverse subjects including STEM, humanities, social sciences, and professional domains

    86.0%

    Self-reported

  • MATH

    #23 / 71

    MATH dataset contains 12,500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem includes full step-by-step solutions and spans multiple difficulty levels (1-5) across seven mathematical subjects including Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Precalculus.

    77.0%

    Self-reported

  • IFEval

    #10 / 70

    Instruction-Following Evaluation (IFEval) benchmark for large language models, focusing on verifiable instructions with 25 types of instructions and around 500 prompts containing one or more verifiable constraints

    92.1%

    Self-reported

  • HumanEval

    #21 / 66

    A benchmark that measures functional correctness for synthesizing programs from docstrings, consisting of 164 original programming problems assessing language comprehension, algorithms, and simple mathematics

    88.4%

    Self-reported

  • MGSM

    #5 / 31

    MGSM (Multilingual Grade School Math) is a benchmark of grade-school math problems. Contains 250 grade-school math problems manually translated from the GSM8K dataset into ten typologically diverse languages: Spanish, French, German, Russian, Chinese, Japanese, Thai, Swahili, Bengali, and Telugu. Evaluates multilingual mathematical reasoning capabilities.

    91.1%

    Self-reported

  • BFCL v2

    #1 / 5

    Berkeley Function Calling Leaderboard (BFCL) v2 is a comprehensive benchmark for evaluating large language models' function calling capabilities. It features 2,251 question-function-answer pairs with enterprise and OSS-contributed functions, addressing data contamination and bias through live, user-contributed scenarios. The benchmark evaluates AST accuracy, executable accuracy, irrelevance detection, and relevance detection across multiple programming languages (Python, Java, JavaScript) and includes complex real-world function calling scenarios with multi-lingual prompts.

    77.3%

    Self-reported

  • MBPP EvalPlus

    #2 / 2

    MBPP (Mostly Basic Python Problems) is a benchmark of 974 crowd-sourced Python programming problems designed to be solvable by entry-level programmers. EvalPlus extends MBPP with significantly more test cases (35x) for more rigorous evaluation of LLM-synthesized code, providing high-quality and precise evaluation.

    87.6%

    Unverified