Skip to content

Microsoft mit Released Dec 12, 2024 Updated

Phi 4

ToolsGuide Scores, rankings, test results and price.

Model summary

  • Text #302 / 416

    156 / 1,000

    ToolsGuide Score· votes 154, tests 158

  • Price per 1M tokens

    In $0.07· Out $0.14

    $0.07

    Input · median text model $0.43

Summary

Phi 4 is Microsoft's model, released Dec 12, 2024. It's ranked in 1 category and does best at text: #302 of 416, with a ToolsGuide Score of 156 against a median of 235.

It costs $0.07 per million input tokens and $0.14 per million output tokens, below the median text model ($0.43).

It has published results on 13 tests.

Specifications

Creator
Microsoft
Licence
mit
Released
Dec 12, 2024
Context window
16K tokens
Input price
$0.07 / 1M
Output price
$0.14 / 1M
Tests reported
13
Settings ranked
phi-4

Model comparison

Against the ten best text models.

Text score

ToolsGuide Score out of 1,000 · Higher is better

Input price

USD per 1M input tokens · Lower is better

Leaderboards

Where Phi 4 sits among the top 20 in each category.

Text

#302 of 416 · ToolsGuide Score out of 1,000, where 1,000 is a flawless model

Full text leaderboard →

By task

Scored from votes on each kind of prompt alone, on the same 1,000-point scale.

Score against price

Text · the 10 priced models ranked closest · up and to the left is better value

Against its nearest rivals

Text prompts by kind · ToolsGuide Score from votes · hover for values

Text

phi-4 · hover a bar for its rank

Test results

Every test Phi 4 has a published result on, out of 100%, with its rank among the models that took it.

  • GPQA

    #186 / 256

    Graduate-level science questions written to be hard to look up.

    56.1%

    Self-reported

  • MMLU-Pro

    #92 / 142

    Broad knowledge and reasoning across 14 subjects, ten answer choices each.

    70.4%

    Self-reported

  • MMLU

    #47 / 103

    Massive Multitask Language Understanding benchmark testing knowledge across 57 diverse subjects including STEM, humanities, social sciences, and professional domains

    84.8%

    Self-reported

  • MATH

    #19 / 71

    MATH dataset contains 12,500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem includes full step-by-step solutions and spans multiple difficulty levels (1-5) across seven mathematical subjects including Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Precalculus.

    80.4%

    Self-reported

  • IFEval

    #67 / 70

    Instruction-Following Evaluation (IFEval) benchmark for large language models, focusing on verifiable instructions with 25 types of instructions and around 500 prompts containing one or more verifiable constraints

    63.0%

    Self-reported

  • HumanEval

    #41 / 66

    A benchmark that measures functional correctness for synthesizing programs from docstrings, consisting of 164 original programming problems assessing language comprehension, algorithms, and simple mathematics

    82.6%

    Self-reported

  • SimpleQA

    #45 / 47

    SimpleQA is a factuality benchmark developed by OpenAI that measures the short-form factual accuracy of large language models. The benchmark contains 4,326 short, fact-seeking questions that are adversarially collected and designed to have single, indisputable answers. Questions cover diverse topics from science and technology to entertainment, and the benchmark also measures model calibration by evaluating whether models know what they know.

    3.0%

    Self-reported

  • LiveBench

    #36 / 38

    LiveBench is a challenging, contamination-limited LLM benchmark that addresses test set contamination by releasing new questions monthly based on recently-released datasets, arXiv papers, news articles, and IMDb movie synopses. It comprises tasks across math, coding, reasoning, language, instruction following, and data analysis with verifiable, objective ground-truth answers.

    47.6%

    Self-reported · In the text score

  • MGSM

    #19 / 31

    MGSM (Multilingual Grade School Math) is a benchmark of grade-school math problems. Contains 250 grade-school math problems manually translated from the GSM8K dataset into ten typologically diverse languages: Spanish, French, German, Russian, Chinese, Japanese, Thai, Swahili, Bengali, and Telugu. Evaluates multilingual mathematical reasoning capabilities.

    80.6%

    Self-reported

  • DROP

    #19 / 30

    DROP (Discrete Reasoning Over Paragraphs) is a reading comprehension benchmark requiring discrete reasoning over paragraph content. It contains crowdsourced, adversarially-created questions that require resolving references and performing discrete operations like addition, counting, or sorting, demanding comprehensive paragraph understanding beyond paraphrase-and-entity-typing shortcuts.

    75.5%

    Self-reported

  • Arena Hard

    #10 / 27

    Arena-Hard-Auto is an automatic evaluation benchmark for instruction-tuned LLMs consisting of 500 challenging real-world prompts curated by BenchBuilder. It includes open-ended software engineering problems, mathematical questions, and creative writing tasks. The benchmark uses LLM-as-a-Judge methodology with GPT-4.1 and Gemini-2.5 as automatic judges to approximate human preference. Arena-Hard achieves 98.6% correlation with human preference rankings and provides 3x higher separation of model performances compared to MT-Bench, making it highly effective for distinguishing between models of similar quality.

    75.4%

    Self-reported

  • HumanEval+

    #5 / 10

    Enhanced version of HumanEval that extends the original test cases by 80x using EvalPlus framework for rigorous evaluation of LLM-synthesized code functional correctness, detecting previously undetected wrong code

    82.8%

    Self-reported

  • PhiBench

    #3 / 3

    PhiBench is an internal benchmark designed to evaluate diverse skills and reasoning abilities of language models, covering a wide range of tasks including coding (debugging, extending incomplete code, explaining code snippets) and mathematics (identifying proof errors, generating related problems). Created by Microsoft's research team to address limitations of standard academic benchmarks and guide the development of the Phi-4 model.

    56.2%

    Self-reported