Skip to content

OpenAI Proprietary Released Sep 12, 2024 Updated

o1-mini

ToolsGuide Scores, rankings, test results and price.

Model summary

  • Text #238 / 416

    205 / 1,000

    ToolsGuide Score· votes 209, tests 197

  • Price per 1M tokens

    No listed price

Summary

o1-mini is OpenAI's model, released Sep 12, 2024. It's ranked in 1 category and does best at text: #238 of 416, with a ToolsGuide Score of 205 against a median of 235.

It has published results on 6 tests and has the top score on 1 of them, including SuperGLUE.

Specifications

Creator
OpenAI
Licence
Proprietary
Released
Sep 12, 2024
Context window
–
Input price
–
Output price
–
Tests reported
6
Settings ranked
o1-mini

Model comparison

Against the ten best text models.

Text score

ToolsGuide Score out of 1,000 · Higher is better

Input price

USD per 1M input tokens · Lower is better

Leaderboards

Where o1-mini sits among the top 20 in each category.

Text

#238 of 416 · ToolsGuide Score out of 1,000, where 1,000 is a flawless model

Full text leaderboard →

By task

Scored from votes on each kind of prompt alone, on the same 1,000-point scale.

Against its nearest rivals

Text prompts by kind · ToolsGuide Score from votes · hover for values

Text

o1-mini · hover a bar for its rank

Test results

Every test o1-mini has a published result on, out of 100%, with its rank among the models that took it.

  • GPQA

    #179 / 256

    Graduate-level science questions written to be hard to look up.

    60.0%

    Self-reported

  • MMLU

    #42 / 103

    Massive Multitask Language Understanding benchmark testing knowledge across 57 diverse subjects including STEM, humanities, social sciences, and professional domains

    85.2%

    Self-reported

  • HumanEval

    #7 / 66

    A benchmark that measures functional correctness for synthesizing programs from docstrings, consisting of 164 original programming problems assessing language comprehension, algorithms, and simple mathematics

    92.4%

    Self-reported

  • MATH-500

    #29 / 34

    MATH-500 is a subset of the MATH dataset containing 500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem includes full step-by-step solutions and spans multiple difficulty levels across seven mathematical subjects including Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Precalculus.

    90.0%

    Self-reported

  • Cybersecurity CTFs

    #3 / 3

    Cybersecurity Capture the Flag (CTF) benchmark for evaluating LLMs in offensive security challenges. Contains diverse cybersecurity tasks including cryptography, web exploitation, binary analysis, and forensics to assess AI capabilities in cybersecurity problem-solving.

    28.7%

    Self-reported

  • SuperGLUE

    #1 / 1

    SuperGLUE is a new benchmark styled after GLUE with a new set of more difficult language understanding tasks, improved resources, and a new public leaderboard. It includes 8 primary tasks: BoolQ (Boolean Questions), CB (CommitmentBank), COPA (Choice of Plausible Alternatives), MultiRC (Multi-Sentence Reading Comprehension), ReCoRD (Reading Comprehension with Commonsense Reasoning), RTE (Recognizing Textual Entailment), WiC (Word-in-Context), and WSC (Winograd Schema Challenge). The benchmark evaluates diverse language understanding capabilities including reading comprehension, commonsense reasoning, causal reasoning, coreference resolution, textual entailment, and word sense disambiguation across multiple domains.

    75.0%

    Unverified