Mistral mistral_research_license Released Jul 24, 2024 Updated
Mistral Large 2
ToolsGuide Scores, rankings, test results and price.
Model summary
- Text #256 / 419
201 / 1,000
ToolsGuide Score· votes 190, tests 230
- Price per 1M tokens
No listed price
Summary
Mistral Large 2 is Mistral's model, released Jul 24, 2024. It's ranked in 1 category and does best at text: #256 of 419, with a ToolsGuide Score of 201 against a median of 252.
It has published results on 5 tests and has the top score on 1 of them, including MMLU French.
Specifications
- Creator
- Mistral
- Licence
- mistral_research_license
- Released
- Jul 24, 2024
- Context window
- –
- Input price
- –
- Output price
- –
- Tests reported
- 5
- Settings ranked
- Mistral Large 2 (2407)
Model comparison
Against the ten best text models.
Text score
ToolsGuide Score out of 1,000 · Higher is better
Input price
USD per 1M input tokens · Lower is better
Leaderboards
Where Mistral Large 2 sits among the top 20 in each category.
Text
#256 of 419 · ToolsGuide Score out of 1,000, where 1,000 is a flawless model
By task
Scored from votes on each kind of prompt alone, on the same 1,000-point scale.
Against its nearest rivals
Text prompts by kind · ToolsGuide Score from votes · hover for values
- Mistral Large 2
-
Gemini 1.5 Flash (002) -
Llama 3.1 70B Instruct
Text
Mistral Large 2 (2407) · hover a bar for its rank
Test results
Every test Mistral Large 2 has a published result on, out of 100%, with its rank among the models that took it.
MMLU
#49 / 104Massive Multitask Language Understanding benchmark testing knowledge across 57 diverse subjects including STEM, humanities, social sciences, and professional domains
84.0%
Self-reported
HumanEval
#9 / 66A benchmark that measures functional correctness for synthesizing programs from docstrings, consisting of 164 original programming problems assessing language comprehension, algorithms, and simple mathematics
92.0%
Self-reported
GSM8k
#18 / 48Grade School Math 8K, a dataset of 8.5K high-quality linguistically diverse grade school math word problems requiring multi-step reasoning and elementary arithmetic operations.
93.0%
Self-reported
MT-Bench
#7 / 13MT-Bench is a challenging multi-turn benchmark that measures the ability of large language models to engage in coherent, informative, and engaging conversations. It uses strong LLMs as judges for scalable and explainable evaluation of multi-turn dialogue capabilities.
0.9%
Self-reported
MMLU French
#1 / 1French language variant of the Massive Multitask Language Understanding benchmark, evaluating language models across 57 tasks including elementary mathematics, US history, computer science, law, and other professional and academic subjects. This multilingual version tests model performance in French.
82.8%
Unverified