Nova 2 Lite

Leaderboards

Where Nova 2 Lite sits among the top 20 in each category.

By task

Scored from votes on each kind of prompt alone, on the same 1,000-point scale.

Test results

Every test Nova 2 Lite has a published result on, out of 100%, with its rank among the models that took it.

• GPQA

  #105 / 256

  Graduate-level science questions written to be hard to look up.

  79.6%

  Self-reported

• MMLU-Pro

  #56 / 142

  Broad knowledge and reasoning across 14 subjects, ten answer choices each.

  80.9%

  Self-reported

• AIME 2025

  #46 / 123

  Competition maths problems from the 2025 American Invitational Mathematics Exam.

  91.0%

  Self-reported

• SWE-Bench Verified

  #85 / 120

  Fixing real bugs in real GitHub projects, checked by the projects' own tests.

  64.5%

  Self-reported

• LiveCodeBench

  #23 / 76

  LiveCodeBench is a holistic and contamination-free evaluation benchmark for large language models for code. It continuously collects new problems from programming contests (LeetCode, AtCoder, CodeForces) and evaluates four different scenarios: code generation, self-repair, code execution, and test output prediction. Problems are annotated with release dates to enable evaluation on unseen problems released after a model's training cutoff.

  71.0%

  Self-reported

• MMMU-Pro

  #52 / 73

  College-level questions that need both an image and text to answer.

  61.8%

  Self-reported · In the vision score

• IFBench

  #29 / 44

  Instruction Following Benchmark evaluating model's ability to follow complex instructions

  70.8%

  Self-reported

• Multi-Challenge

  #2 / 41

  MultiChallenge is a realistic multi-turn conversation evaluation benchmark that challenges frontier LLMs across four key categories: instruction retention (maintaining instructions throughout conversations), inference memory (recalling and connecting details from previous turns), reliable versioned editing (adapting to evolving instructions during collaborative editing), and self-coherence (avoiding contradictions in responses). The benchmark evaluates models on sustained, contextually complex dialogues across diverse topics including travel planning, technical documentation, and professional communication.

  76.6%

  Self-reported

• MCP Atlas

  #38 / 38

  MCP Atlas is a benchmark for evaluating AI models on scaled tool use capabilities, measuring how well models can coordinate and utilize multiple tools across complex multi-step tasks.

  24.6%

  Self-reported

• Tau2 Telecom

  #23 / 36

  τ²-Bench telecom domain evaluates conversational agents in a dual-control environment modeled as a Dec-POMDP, where both agent and user use tools in shared telecommunications troubleshooting scenarios that test coordination and communication capabilities.

  76.0%

  Self-reported

• Tau2 Retail

  #15 / 27

  τ²-bench retail domain evaluates conversational AI agents in customer service scenarios within a dual-control environment where both agent and user can interact with tools. Tests tool-agent-user interaction, rule adherence, and task consistency in retail customer support contexts.

  76.5%

  Self-reported

• Tau2 Airline

  #9 / 25

  TAU2 airline domain benchmark for evaluating conversational agents in dual-control environments where both AI agents and users interact with tools in airline customer service scenarios. Tests agent coordination, communication, and ability to guide user actions in tasks like flight booking, modifications, cancellations, and refunds.

  64.8%

  Self-reported

• Terminal-Bench

  #17 / 25

  Terminal-Bench is a benchmark for testing AI agents in real terminal environments. It evaluates how well agents can handle real-world, end-to-end tasks autonomously, including compiling code, training models, setting up servers, system administration, security tasks, data science workflows, and cybersecurity vulnerabilities. The benchmark consists of a dataset of ~100 hand-crafted, human-verified tasks and an execution harness that connects language models to a terminal sandbox.

  32.5%

  Self-reported

• BFCL-V4

  #12 / 21

  Berkeley Function Calling Leaderboard V4 (BFCL-V4) evaluates LLMs on their ability to accurately call functions and APIs, including simple, multiple, parallel, and nested function calls across diverse programming scenarios.

  60.3%

  Self-reported

• ScreenSpot

  #16 / 16

  ScreenSpot is the first realistic GUI grounding benchmark that encompasses mobile, desktop, and web environments. The dataset comprises over 1,200 instructions from iOS, Android, macOS, Windows and Web environments, along with annotated element types (text and icon/widget), designed to evaluate visual GUI agents' ability to accurately locate screen elements based on natural language instructions.

  83.3%

  Self-reported

• OCRBench_V2

  #8 / 8

  OCRBench v2: Enhanced large-scale bilingual benchmark for evaluating Large Multimodal Models on visual text localization and reasoning with 10,000 human-verified question-answering pairs across 8 core OCR capabilities

  56.1%

  Self-reported

• QVHighlights

  #1 / 3

  QVHighlights is a video moment retrieval benchmark for detecting moments and highlights in videos via natural language queries. Given a query, the model must localize the start and end times of relevant moments in the video, evaluated using metrics such as Recall@1 at a 0.5 IoU threshold.

  77.2%

  Self-reported

• RealKIE-FCC

  #2 / 3

  RealKIE-FCC is a key information extraction benchmark drawn from real enterprise documents (FCC filings), part of the RealKIE suite of five novel datasets for enterprise key information extraction. Models must convert documents to markdown and extract structured fields against a specified JSON schema. Nova 2 reports results on a human-verified version of the dataset.

  62.1%

  Self-reported

• LongCodeBench

  #1 / 2

  LongCodeBench evaluates the code understanding and comprehension abilities of large language models at very long context windows, scaling up to 1M tokens. It tests whether models can reason about extensive codebases provided in a single prompt by answering multiple-choice questions about the code.

  84.0%

  Unverified