IBM Granite 4.2 8B

Leaderboards

Where IBM Granite 4.2 8B sits among the top 20 in each category.

By task

Scored from votes on each kind of prompt alone, on the same 1,000-point scale.

Test results

Every test IBM Granite 4.2 8B has a published result on, out of 100%, with its rank among the models that took it.

• GPQA

  #173 / 256

  Graduate-level science questions written to be hard to look up.

  64.1%

  Self-reported

• MMLU-Pro

  #84 / 142

  Broad knowledge and reasoning across 14 subjects, ten answer choices each.

  74.0%

  Self-reported

• AIME 2025

  #62 / 123

  Competition maths problems from the 2025 American Invitational Mathematics Exam.

  86.7%

  Self-reported

• SWE-Bench Verified

  #105 / 120

  Fixing real bugs in real GitHub projects, checked by the projects' own tests.

  47.7%

  Self-reported

• LiveCodeBench v6

  #36 / 67

  LiveCodeBench is a holistic and contamination-free evaluation benchmark for large language models for code. It continuously collects new problems from programming contests (LeetCode, AtCoder, CodeForces) and evaluates four different scenarios: code generation, self-repair, code execution, and test output prediction. Problems are annotated with release dates to enable evaluation on unseen problems released after a model's training cutoff.

  73.2%

  Self-reported

• SWE-Bench Pro

  #61 / 61

  Harder, longer software engineering tasks from real codebases.

  19.1%

  Self-reported · In the coding score

• SWE-bench Multilingual

  #48 / 49

  A multilingual benchmark for issue resolving in software engineering that covers Java, TypeScript, JavaScript, Go, Rust, C, and C++. Contains 1,632 high-quality instances carefully annotated from 2,456 candidates by 68 expert annotators, designed to evaluate Large Language Models across diverse software ecosystems beyond Python.

  30.8%

  Self-reported

• Terminal-Bench 2.1

  #45 / 46

  Terminal-Bench 2.1 is an updated release of the Terminal-Bench benchmark that tests AI agents' ability to operate a computer via the terminal. It evaluates how well models handle real-world, end-to-end tasks autonomously, including compiling code, training models, setting up servers, system administration, data science workflows, and security tasks.

  20.6%

  Self-reported

• IFBench

  #10 / 44

  Instruction Following Benchmark evaluating model's ability to follow complex instructions

  79.3%

  Self-reported

• SciCode

  #25 / 30

  SciCode is a research coding benchmark curated by scientists that challenges language models to code solutions for scientific problems. It contains 338 subproblems decomposed from 80 challenging main problems across 16 natural science sub-fields including mathematics, physics, chemistry, biology, and materials science. Problems require knowledge recall, reasoning, and code synthesis skills.

  36.1%

  Self-reported

• HMMT25

  #14 / 28

  Harvard-MIT Mathematics Tournament 2025 - A prestigious student-organized mathematics competition for high school students featuring two tournaments (November 2025 at MIT and February 2026 at Harvard) with individual tests, team rounds, and guts rounds

  78.3%

  Self-reported

• BFCL-V4

  #17 / 21

  Berkeley Function Calling Leaderboard V4 (BFCL-V4) evaluates LLMs on their ability to accurately call functions and APIs, including simple, multiple, parallel, and nested function calls across diverse programming scenarios.

  52.4%

  Self-reported

• Arena-Hard v2

  #10 / 19

  Arena-Hard-Auto v2 is a challenging benchmark consisting of 500 carefully curated prompts sourced from Chatbot Arena and WildChat-1M, designed to evaluate large language models on real-world user queries. The benchmark covers diverse domains including open-ended software engineering problems, mathematics, creative writing, and technical problem-solving. It uses LLM-as-a-Judge for automatic evaluation, achieving 98.6% correlation with human preference rankings while providing 3x higher separation of model performances compared to MT-Bench. The benchmark emphasizes prompt specificity, complexity, and domain knowledge to better distinguish between model capabilities.

  65.2%

  Self-reported

• Bird-SQL (dev)

  #7 / 9

  BIRD (BIg Bench for LaRge-scale Database Grounded Text-to-SQLs) is a comprehensive text-to-SQL benchmark containing 12,751 question-SQL pairs across 95 databases (33.4 GB total) spanning 37+ professional domains. It evaluates large language models' ability to convert natural language to executable SQL queries in real-world scenarios with complex database schemas and dirty data.

  41.1%

  Self-reported

• TAU3-Bench

  #6 / 8

  TAU3-Bench is a benchmark for evaluating general-purpose agent capabilities, testing models on multi-turn interactions with simulated user models, retrieval, and complex decision-making scenarios.

  58.1%

  Self-reported

• ProfBench

  #3 / 4

  ProfBench evaluates models on professional-domain reasoning and knowledge-work tasks, including search-augmented question answering across expert fields.

  41.2%

  Self-reported

• RULER 128k

  #3 / 4

  RULER 128k evaluates the official 13-task RULER v1 suite at a 131072-token context budget.

  71.4%

  Self-reported

• RULER 64k

  #3 / 4

  RULER 64k evaluates the official 13-task RULER v1 suite at a 65536-token context budget.

  81.0%

  Self-reported