Skip to content

xAI Proprietary Released Aug 13, 2024 Updated

Grok-2 mini

ToolsGuide Scores, rankings, test results and price.

Model summary

  • Text #259 / 419

    200 / 1,000

    ToolsGuide Score· votes 185, tests 238

  • Price per 1M tokens

    No listed price

Summary

Grok-2 mini is xAI's model, released Aug 13, 2024. It's ranked in 1 category and does best at text: #259 of 419, with a ToolsGuide Score of 200 against a median of 252.

It has published results on 8 tests.

Specifications

Creator
xAI
Licence
Proprietary
Released
Aug 13, 2024
Context window
–
Input price
–
Output price
–
Tests reported
8
Settings ranked
Grok 2 mini (2024-08-13)

Model comparison

Against the ten best text models.

Text score

ToolsGuide Score out of 1,000 · Higher is better

Input price

USD per 1M input tokens · Lower is better

Leaderboards

Where Grok-2 mini sits among the top 20 in each category.

Text

#259 of 419 · ToolsGuide Score out of 1,000, where 1,000 is a flawless model

Full text leaderboard →

By task

Scored from votes on each kind of prompt alone, on the same 1,000-point scale.

Against its nearest rivals

Text prompts by kind · ToolsGuide Score from votes · hover for values

Text

Grok 2 mini (2024-08-13) · hover a bar for its rank

Test results

Every test Grok-2 mini has a published result on, out of 100%, with its rank among the models that took it.

  • GPQA

    #203 / 265

    Graduate-level science questions written to be hard to look up.

    51.0%

    Self-reported

  • MMLU-Pro

    #88 / 142

    Broad knowledge and reasoning across 14 subjects, ten answer choices each.

    72.0%

    Self-reported

  • MMLU

    #33 / 104

    Massive Multitask Language Understanding benchmark testing knowledge across 57 diverse subjects including STEM, humanities, social sciences, and professional domains

    86.2%

    Self-reported

  • MATH

    #32 / 71

    MATH dataset contains 12,500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem includes full step-by-step solutions and spans multiple difficulty levels (1-5) across seven mathematical subjects including Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Precalculus.

    73.0%

    Self-reported

  • HumanEval

    #33 / 66

    A benchmark that measures functional correctness for synthesizing programs from docstrings, consisting of 164 original programming problems assessing language comprehension, algorithms, and simple mathematics

    85.7%

    Self-reported

  • MMMU

    #43 / 66

    MMMU (Massive Multi-discipline Multimodal Understanding) is a benchmark designed to evaluate multimodal models on college-level subject knowledge and deliberate reasoning. Contains 11.5K meticulously collected multimodal questions from college exams, quizzes, and textbooks, covering six core disciplines: Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering across 30 subjects and 183 subfields.

    63.2%

    Self-reported

  • MathVista

    #18 / 40

    MathVista evaluates mathematical reasoning of foundation models in visual contexts. It consists of 6,141 examples derived from 28 existing multimodal datasets and 3 newly created datasets (IQTest, FunctionQA, and PaperQA), combining challenges from diverse mathematical and visual tasks to assess models' ability to understand complex figures and perform rigorous reasoning.

    68.1%

    Self-reported

  • DocVQA

    #13 / 28

    A dataset for Visual Question Answering on document images containing 50,000 questions defined on 12,000+ document images. The benchmark tests AI's ability to understand document structure and content, requiring models to comprehend document layout and perform information retrieval to answer questions about document images.

    93.2%

    Self-reported