Knowledge · Benchmark profile

Measuring Short-Form Factuality in Large Language Models

A benchmark that evaluates the ability of language models to answer short, fact-seeking questions accurately. Focuses on factual correctness rather than reasoning complexity.

Reference2024Active9 models

Data verified 21 Jul 2026 · Methodology 1.6.0

Benchmark score on Measuring Short-Form Factuality in Large Language Models

Leaderboard · 9 models

DeepSeek V4 Pro (Max) leads at 57.9%, followed by DeepSeek V4 Pro Base (55.2%) and DeepSeek V4 Pro (High) (46.2%).

Display onlyKnowledge2024

#1Open

DeepSeek V4 Pro (Max)

DeepSeek

deepseek-v4-pro-max

57.9%

Overall — · Context 1M tokens

#2Open

DeepSeek V4 Pro Base

DeepSeek

deepseek-v4-pro-base

55.2%

Overall — · Context 1M tokens

#3Open

DeepSeek V4 Pro (High)

DeepSeek

deepseek-v4-pro-high

46.2%

Overall — · Context 1M tokens

Visual analysis

See the field.

Switch between model placement, score distribution and descriptive provider averages. Every view uses the same sourced leaderboard.

ViewModelsColours

DeepSeekMicrosoft

Measuring Short-Form Factuality in Large Language Models leaderboard

9 ranked models · higher is better

View accessible chart data

Accessible data for Measuring Short-Form Factuality in Large Language Models leaderboard
Model	Rank	Provider	Score
DeepSeek V4 Pro (Max)	#1	DeepSeek	57.9%
DeepSeek V4 Pro Base	#2	DeepSeek	55.2%
DeepSeek V4 Pro (High)	#3	DeepSeek	46.2%
DeepSeek V4 Pro	#4	DeepSeek	45%
DeepSeek V4 Flash (Max)	#5	DeepSeek	34.1%
MAI-Thinking-1	#6	Microsoft	31%
DeepSeek V4 Flash Base	#7	DeepSeek	30.1%
DeepSeek V4 Flash (High)	#8	DeepSeek	28.9%
DeepSeek V4 Flash	#9	DeepSeek	23.1%

Benchmark score table (9 models)

One best score per model · higher is better

Measuring Short-Form Factuality in Large Language Models leaderboard ranked by score
Rank	Model	Provider	License	Evidence use	Score
#1	DeepSeek V4 Pro (Max) deepseek-v4-pro-max	DeepSeek	open	Estimated reference	57.9%
#2	DeepSeek V4 Pro Base deepseek-v4-pro-base	DeepSeek	open	Estimated reference	55.2%
#3	DeepSeek V4 Pro (High) deepseek-v4-pro-high	DeepSeek	open	Estimated reference	46.2%
#4	DeepSeek V4 Pro deepseek-v4-pro	DeepSeek	open	Reference only	45%
#5	DeepSeek V4 Flash (Max) deepseek-v4-flash-max	DeepSeek	open	Estimated reference	34.1%
#6	MAI-Thinking-1 mai-thinking-1	Microsoft	closed	Estimated reference	31%
#7	DeepSeek V4 Flash Base deepseek-v4-flash-base	DeepSeek	open	Estimated reference	30.1%
#8	DeepSeek V4 Flash (High) deepseek-v4-flash-high	DeepSeek	open	Estimated reference	28.9%
#9	DeepSeek V4 Flash deepseek-v4-flash	DeepSeek	open	Estimated reference	23.1%

The top of this snapshot is led by DeepSeek V4 Pro (Max) at 57.9%; third place is 11.7 points behind. The top-9 spread is 34.8 points.

Compare top models

DeepSeek V4 Pro (Max) vs DeepSeek V4 Pro Base DeepSeek V4 Pro Base vs DeepSeek V4 Pro (High)DeepSeek V4 Pro (High) vs DeepSeek V4 Pro DeepSeek V4 Pro vs DeepSeek V4 Flash (Max)

About Measuring Short-Form Factuality in Large Language Models

Definition and scoring

Organisation: Jason Wei, Najoung Kim, Hyung Won Chung, Yu-An Chung, Siddhartha Papay, Yifeng Lu, Hannaneh Hajishirzi, Luke Zettlemoyer
Category: Knowledge
Version: 2024
Direction: higher is better
Ranking use: Reference
Contamination risk: Unknown

A benchmark that evaluates the ability of language models to answer short, fact-seeking questions accurately. Focuses on factual correctness rather than reasoning complexity. Results stay tied to the exact model variant and evaluation system. Multiple systems for the same model use the best published score on this page; overall Lumina scoring uses the median of ranking-eligible rows.

Open benchmark source ↗

FAQ

Quick answers

What does Measuring Short-Form Factuality in Large Language Models measure?

A benchmark that evaluates the ability of language models to answer short, fact-seeking questions accurately. Focuses on factual correctness rather than reasoning complexity.

Which model scores highest on Measuring Short-Form Factuality in Large Language Models?

DeepSeek V4 Pro (Max) by DeepSeek currently leads with 57.9%.

How many models are evaluated on Measuring Short-Form Factuality in Large Language Models?

9 models in the LuminaBench cohort have a qualifying score on this benchmark.

Artificial Analysis Openness Index

2026 · 10 results · Reference

Artificial Analysis MMLU-Pro

2026 · 4 results · Reference

Massive Multitask Language Understanding

2020 · 8 results · Reference

Graduate-Level Google-Proof Q&A

2023 · 70 results · Reference

SuperGPQA: Scaling LLM Evaluation Across 285 Graduate Disciplines

2025 · 19 results · Reference

Massive Multitask Language Understanding Professional

2024 · 42 results · Reference

Knowledge · Benchmark profile

Measuring Short-Form Factuality in Large Language Models

A benchmark that evaluates the ability of language models to answer short, fact-seeking questions accurately. Focuses on factual correctness rather than reasoning complexity.

Reference2024Active9 models

Data verified 21 Jul 2026 · Methodology 1.6.0

Benchmark score on Measuring Short-Form Factuality in Large Language Models

Leaderboard · 9 models

DeepSeek V4 Pro (Max) leads at 57.9%, followed by DeepSeek V4 Pro Base (55.2%) and DeepSeek V4 Pro (High) (46.2%).

Display onlyKnowledge2024

#1Open

DeepSeek V4 Pro (Max)

DeepSeek

deepseek-v4-pro-max

57.9%

Overall — · Context 1M tokens

#2Open

DeepSeek V4 Pro Base

DeepSeek

deepseek-v4-pro-base

55.2%

Overall — · Context 1M tokens

#3Open

DeepSeek V4 Pro (High)

DeepSeek

deepseek-v4-pro-high

46.2%

Overall — · Context 1M tokens

Visual analysis

See the field.

Switch between model placement, score distribution and descriptive provider averages. Every view uses the same sourced leaderboard.

ViewModelsColours

DeepSeekMicrosoft

Measuring Short-Form Factuality in Large Language Models leaderboard

9 ranked models · higher is better

View accessible chart data

Accessible data for Measuring Short-Form Factuality in Large Language Models leaderboard
Model	Rank	Provider	Score
DeepSeek V4 Pro (Max)	#1	DeepSeek	57.9%
DeepSeek V4 Pro Base	#2	DeepSeek	55.2%
DeepSeek V4 Pro (High)	#3	DeepSeek	46.2%
DeepSeek V4 Pro	#4	DeepSeek	45%
DeepSeek V4 Flash (Max)	#5	DeepSeek	34.1%
MAI-Thinking-1	#6	Microsoft	31%
DeepSeek V4 Flash Base	#7	DeepSeek	30.1%
DeepSeek V4 Flash (High)	#8	DeepSeek	28.9%
DeepSeek V4 Flash	#9	DeepSeek	23.1%

Benchmark score table (9 models)

One best score per model · higher is better

Measuring Short-Form Factuality in Large Language Models leaderboard ranked by score
Rank	Model	Provider	License	Evidence use	Score
#1	DeepSeek V4 Pro (Max) deepseek-v4-pro-max	DeepSeek	open	Estimated reference	57.9%
#2	DeepSeek V4 Pro Base deepseek-v4-pro-base	DeepSeek	open	Estimated reference	55.2%
#3	DeepSeek V4 Pro (High) deepseek-v4-pro-high	DeepSeek	open	Estimated reference	46.2%
#4	DeepSeek V4 Pro deepseek-v4-pro	DeepSeek	open	Reference only	45%
#5	DeepSeek V4 Flash (Max) deepseek-v4-flash-max	DeepSeek	open	Estimated reference	34.1%
#6	MAI-Thinking-1 mai-thinking-1	Microsoft	closed	Estimated reference	31%
#7	DeepSeek V4 Flash Base deepseek-v4-flash-base	DeepSeek	open	Estimated reference	30.1%
#8	DeepSeek V4 Flash (High) deepseek-v4-flash-high	DeepSeek	open	Estimated reference	28.9%
#9	DeepSeek V4 Flash deepseek-v4-flash	DeepSeek	open	Estimated reference	23.1%

The top of this snapshot is led by DeepSeek V4 Pro (Max) at 57.9%; third place is 11.7 points behind. The top-9 spread is 34.8 points.

Compare top models

DeepSeek V4 Pro (Max) vs DeepSeek V4 Pro Base DeepSeek V4 Pro Base vs DeepSeek V4 Pro (High)DeepSeek V4 Pro (High) vs DeepSeek V4 Pro DeepSeek V4 Pro vs DeepSeek V4 Flash (Max)

About Measuring Short-Form Factuality in Large Language Models

Definition and scoring

Organisation: Jason Wei, Najoung Kim, Hyung Won Chung, Yu-An Chung, Siddhartha Papay, Yifeng Lu, Hannaneh Hajishirzi, Luke Zettlemoyer
Category: Knowledge
Version: 2024
Direction: higher is better
Ranking use: Reference
Contamination risk: Unknown

Open benchmark source ↗

FAQ

Quick answers

What does Measuring Short-Form Factuality in Large Language Models measure?

A benchmark that evaluates the ability of language models to answer short, fact-seeking questions accurately. Focuses on factual correctness rather than reasoning complexity.

Which model scores highest on Measuring Short-Form Factuality in Large Language Models?

DeepSeek V4 Pro (Max) by DeepSeek currently leads with 57.9%.

How many models are evaluated on Measuring Short-Form Factuality in Large Language Models?

9 models in the LuminaBench cohort have a qualifying score on this benchmark.

Artificial Analysis Openness Index

2026 · 10 results · Reference

Artificial Analysis MMLU-Pro

2026 · 4 results · Reference

Massive Multitask Language Understanding

2020 · 8 results · Reference

Graduate-Level Google-Proof Q&A

2023 · 70 results · Reference

SuperGPQA: Scaling LLM Evaluation Across 285 Graduate Disciplines

2025 · 19 results · Reference

Massive Multitask Language Understanding Professional

2024 · 42 results · Reference

Search LUMINA BENCH

Navigation

Measuring Short-Form Factuality in Large Language Models

Leaderboard · 9 models

DeepSeek V4 Pro (Max)

DeepSeek V4 Pro Base

DeepSeek V4 Pro (High)

See the field.

Measuring Short-Form Factuality in Large Language Models leaderboard

Crop Measuring Short-Form Factuality in Large Language Models leaderboard

Benchmark score table (9 models)

Crop Measuring Short-Form Factuality in Large Language Models benchmark table

Compare top models

Definition and scoring

Quick answers

What does Measuring Short-Form Factuality in Large Language Models measure?

Which model scores highest on Measuring Short-Form Factuality in Large Language Models?

How many models are evaluated on Measuring Short-Form Factuality in Large Language Models?

Measuring Short-Form Factuality in Large Language Models

Leaderboard · 9 models

DeepSeek V4 Pro (Max)

DeepSeek V4 Pro Base

DeepSeek V4 Pro (High)

See the field.

Measuring Short-Form Factuality in Large Language Models leaderboard

Crop Measuring Short-Form Factuality in Large Language Models leaderboard

Benchmark score table (9 models)

Crop Measuring Short-Form Factuality in Large Language Models benchmark table

Compare top models

Definition and scoring

Quick answers

What does Measuring Short-Form Factuality in Large Language Models measure?

Which model scores highest on Measuring Short-Form Factuality in Large Language Models?

How many models are evaluated on Measuring Short-Form Factuality in Large Language Models?

Leaderboard · 9 models

DeepSeek V4 Pro (Max)

DeepSeek V4 Pro Base

DeepSeek V4 Pro (High)

See the field.

Benchmark score table (9 models)

Compare top models

Definition and scoring

Quick answers

What does Measuring Short-Form Factuality in Large Language Models measure?

Which model scores highest on Measuring Short-Form Factuality in Large Language Models?

How many models are evaluated on Measuring Short-Form Factuality in Large Language Models?

More Knowledge benchmarks

Leaderboard · 9 models

DeepSeek V4 Pro (Max)

DeepSeek V4 Pro Base

DeepSeek V4 Pro (High)

See the field.

Benchmark score table (9 models)

Compare top models

Definition and scoring

Quick answers

What does Measuring Short-Form Factuality in Large Language Models measure?

Which model scores highest on Measuring Short-Form Factuality in Large Language Models?

How many models are evaluated on Measuring Short-Form Factuality in Large Language Models?

More Knowledge benchmarks