Skip to content

OpenAI Proprietary Released May 13, 2024 Updated

GPT-4o

ToolsGuide Scores, rankings, test results and price.

Model summary

  • Text #235 / 416

    207 / 1,000

    ToolsGuide Score· votes 218, tests 196

  • Vision #107 / 167

    269 / 1,000

    ToolsGuide Score· votes only

  • Price per 1M tokens

    In $2.50· Out $10.00

    $2.50

    Input · median text model $0.43

Summary

GPT-4o is OpenAI's model, released May 13, 2024. It's ranked in 2 categories and does best at text: #235 of 416, with a ToolsGuide Score of 207 against a median of 235.

It also ranks #107 of 167 for vision (269).

Its vision score uses votes only, because it has published fewer than two of the tests used for that category.

It costs $2.50 per million input tokens and $10.00 per million output tokens, about 5.7× the median text model ($0.43).

It has published results on 8 tests.

Specifications

Creator
OpenAI
Licence
Proprietary
Released
May 13, 2024
Context window
128K tokens
Input price
$2.50 / 1M
Output price
$10.00 / 1M
Tests reported
8
Settings ranked
gpt-4o-2024-05-13, GPT-4o (2024-05-13), GPT-4o (2024-08-06)

Model comparison

Against the ten best text models.

Text score

ToolsGuide Score out of 1,000 · Higher is better

Vision score

ToolsGuide Score out of 1,000 · Higher is better

Input price

USD per 1M input tokens · Lower is better

Leaderboards

Where GPT-4o sits among the top 20 in each category.

Text

#235 of 416 · ToolsGuide Score out of 1,000, where 1,000 is a flawless model

Full text leaderboard →

Vision

#107 of 167 · ToolsGuide Score out of 1,000, where 1,000 is a flawless model

Full vision leaderboard →

By task

Scored from votes on each kind of prompt alone, on the same 1,000-point scale.

Score against price

Text · the 10 priced models ranked closest · up and to the left is better value

Against its nearest rivals

Text prompts by kind · ToolsGuide Score from votes · hover for values

Text

gpt-4o-2024-05-13 · hover a bar for its rank

Vision

GPT-4o (2024-05-13) · hover a bar for its rank

Test results

Every test GPT-4o has a published result on, out of 100%, with its rank among the models that took it.

  • GPQA

    #190 / 256

    Graduate-level science questions written to be hard to look up.

    53.6%

    Self-reported

  • MMLU-Pro

    #87 / 142

    Broad knowledge and reasoning across 14 subjects, ten answer choices each.

    72.6%

    Self-reported

  • MMLU

    #18 / 103

    Massive Multitask Language Understanding benchmark testing knowledge across 57 diverse subjects including STEM, humanities, social sciences, and professional domains

    88.7%

    Self-reported

  • MATH

    #24 / 71

    MATH dataset contains 12,500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem includes full step-by-step solutions and spans multiple difficulty levels (1-5) across seven mathematical subjects including Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Precalculus.

    76.6%

    Self-reported

  • HumanEval

    #12 / 66

    A benchmark that measures functional correctness for synthesizing programs from docstrings, consisting of 164 original programming problems assessing language comprehension, algorithms, and simple mathematics

    90.2%

    Self-reported

  • MathVista

    #24 / 40

    MathVista evaluates mathematical reasoning of foundation models in visual contexts. It consists of 6,141 examples derived from 28 existing multimodal datasets and 3 newly created datasets (IQTest, FunctionQA, and PaperQA), combining challenges from diverse mathematical and visual tasks to assess models' ability to understand complex figures and perform rigorous reasoning.

    63.8%

    Self-reported

  • MGSM

    #9 / 31

    MGSM (Multilingual Grade School Math) is a benchmark of grade-school math problems. Contains 250 grade-school math problems manually translated from the GSM8K dataset into ten typologically diverse languages: Spanish, French, German, Russian, Chinese, Japanese, Thai, Swahili, Bengali, and Telugu. Evaluates multilingual mathematical reasoning capabilities.

    90.5%

    Self-reported

  • DROP

    #8 / 30

    DROP (Discrete Reasoning Over Paragraphs) is a reading comprehension benchmark requiring discrete reasoning over paragraph content. It contains crowdsourced, adversarially-created questions that require resolving references and performing discrete operations like addition, counting, or sorting, demanding comprehensive paragraph understanding beyond paraphrase-and-entity-typing shortcuts.

    83.4%

    Self-reported