Beam 1.0

Leaderboards

Where Beam 1.0 sits among the top 20 in each category.

By task

Scored from votes on each kind of prompt alone, on the same 1,000-point scale.

Test results

Every test Beam 1.0 has a published result on, out of 100%, with its rank among the models that took it.

• GPQA

  #25 / 265

  Graduate-level science questions written to be hard to look up.

  90.6%

  Self-reported

• SWE-Bench Verified

  #7 / 123

  Fixing real bugs in real GitHub projects, checked by the projects' own tests.

  81.6%

  Self-reported

• Humanity's Last Exam

  #48 / 115

  Humanity's Last Exam (HLE) is a multi-modal academic benchmark with 2,500 questions across mathematics, humanities, and natural sciences, designed to test LLM capabilities at the frontier of human knowledge with unambiguous, verifiable solutions

  36.0%

  Self-reported · In the text score· In the vision score

• LiveCodeBench

  #5 / 77

  LiveCodeBench is a holistic and contamination-free evaluation benchmark for large language models for code. It continuously collects new problems from programming contests (LeetCode, AtCoder, CodeForces) and evaluates four different scenarios: code generation, self-repair, code execution, and test output prediction. Problems are annotated with release dates to enable evaluation on unseen problems released after a model's training cutoff.

  88.3%

  Self-reported

• BrowseComp

  #29 / 73

  Finding hard-to-locate facts by browsing the web.

  76.8%

  Self-reported · In the agents score

• SWE-Bench Pro

  #7 / 63

  Harder, longer software engineering tasks from real codebases.

  66.1%

  Self-reported · In the coding score

• Terminal-Bench 2.1

  #24 / 53

  Terminal-Bench 2.1 is an updated release of the Terminal-Bench benchmark that tests AI agents' ability to operate a computer via the terminal. It evaluates how well models handle real-world, end-to-end tasks autonomously, including compiling code, training models, setting up servers, system administration, data science workflows, and security tasks.

  80.1%

  Self-reported

• DeepSWE 1.1

  #46 / 52

  DeepSWE 1.1 evaluates software engineering agents using the mini-swe-agent harness.

  43.3%

  Self-reported · In the coding score

• SWE-bench Multilingual

  #10 / 50

  A multilingual benchmark for issue resolving in software engineering that covers Java, TypeScript, JavaScript, Go, Rust, C, and C++. Contains 1,632 high-quality instances carefully annotated from 2,456 candidates by 68 expert annotators, designed to evaluate Large Language Models across diverse software ecosystems beyond Python.

  78.3%

  Self-reported

• IFBench

  #9 / 46

  Instruction Following Benchmark evaluating model's ability to follow complex instructions

  79.6%

  Self-reported

• MCP Atlas

  #8 / 39

  MCP Atlas is a benchmark for evaluating AI models on scaled tool use capabilities, measuring how well models can coordinate and utilize multiple tools across complex multi-step tasks.

  80.8%

  Self-reported

• SciCode

  #9 / 36

  SciCode is a research coding benchmark curated by scientists that challenges language models to code solutions for scientific problems. It contains 338 subproblems decomposed from 80 challenging main problems across 16 natural science sub-fields including mathematics, physics, chemistry, biology, and materials science. Problems require knowledge recall, reasoning, and code synthesis skills.

  49.2%

  Self-reported

• AA-LCR

  #3 / 31

  Agent Arena Long Context Reasoning benchmark

  78.2%

  Self-reported

• AIME 2026

  #2 / 28

  All 30 problems from the 2026 American Invitational Mathematics Examination (AIME I and AIME II), testing olympiad-level mathematical reasoning with integer answers from 000-999. Used as an AI benchmark to evaluate large language models' ability to solve complex mathematical problems requiring multi-step logical deductions and structured symbolic reasoning.

  97.7%

  Self-reported

• AutomationBench

  #12 / 27

  AutomationBench is a tool-use benchmark that evaluates AI agents on automating real-world workflows, testing their ability to orchestrate tools and complete multi-step automation tasks.

  39.3%

  Self-reported · In the agents score

• LongBench v2

  #1 / 25

  LongBench v2 is a benchmark designed to assess the ability of LLMs to handle long-context problems requiring deep understanding and reasoning across real-world multitasks. It consists of 503 challenging multiple-choice questions with contexts ranging from 8k to 2M words across six major task categories: single-document QA, multi-document QA, long in-context learning, long-dialogue history understanding, code repository understanding, and long structured data understanding.

  67.5%

  Self-reported

• Tau3 Banking

  #2 / 20

  τ³-Bench banking domain evaluates agentic models on multi-turn, tool-using customer-support scenarios in a simulated retail banking environment.

  36.1%

  Self-reported

• DeepSearchQA

  #8 / 12

  DeepSearchQA is a benchmark for evaluating deep search and question-answering capabilities, testing models' ability to perform multi-hop reasoning and information retrieval across complex knowledge domains.

  80.5%

  Self-reported

• CritPT

  #3 / 10

  CritPT is a challenging reasoning benchmark reported by Qwen for evaluating frontier mathematical and critical problem-solving capability.

  16.3%

  Self-reported

• SWE Atlas - Codebase QnA

  #7 / 7

  SWE Atlas - Codebase QnA evaluates a model's ability to answer questions about real codebases, measuring repository-level comprehension and the ability to reason about code structure, behavior, and intent across an entire project.

  31.8%

  Self-reported

• Humanity's Last Exam (with tools)

  #6 / 6

  Full multimodal Humanity's Last Exam evaluated with tool use enabled, distinct from its text-only subset.

  51.1%

  Self-reported

• AA-Omniscience Index

  #3 / 5

  AA-Omniscience Index is Artificial Analysis's knowledge-reliability metric. It rewards correct answers, penalizes hallucinations, and does not penalize abstention. Scores range from -100 to 100, where 0 means as many correct as incorrect answers.

  5.4%

  Self-reported