Coding · Benchmark profile

NL2Repo

A repository-understanding benchmark that measures whether models can map natural-language requests onto the right code locations and system changes.

Reference2026Active13 models

Data verified 21 Jul 2026 · Methodology 1.6.0

Benchmark score on NL2Repo

Leaderboard · 13 models

GLM-5.2 leads at 48.9%, followed by Ornith-1.0-397B (48.2%) and Qwen3.7-Max (47.2%).

Display onlyCoding2026

#1Open

GLM-5.2

Z.AI

glm-5-2

48.9%

Overall #25 · Context 1M tokens

#2Open

Ornith-1.0-397B

DeepReinforce AI

ornith-1-0-397b

48.2%

Overall — · Context 256K tokens

#3Closed

Qwen3.7-Max

Alibaba Cloud

qwen3.7-max

47.2%

Overall #37 · Context 1M tokens

Visual analysis

See the field.

Switch between model placement, score distribution and descriptive provider averages. Every view uses the same sourced leaderboard.

ViewModelsColours

Z.AIDeepReinforce AIAlibaba CloudAnthropicMiniMax

NL2Repo leaderboard

13 ranked models · higher is better

View accessible chart data

Accessible data for NL2Repo leaderboard
Model	Rank	Provider	Score
GLM-5.2	#1	Z.AI	48.9%
Ornith-1.0-397B	#2	DeepReinforce AI	48.2%
Qwen3.7-Max	#3	Alibaba Cloud	47.2%
Claude Opus 4.5	#4	Anthropic	43.2%
Qwen 3.6 Max (preview)	#5	Alibaba Cloud	42.9%
GLM-5.1	#6	Z.AI	42.7%
MiniMax M3	#7	MiniMax	42.1%
Qwen3.7-Plus	#8	Alibaba Cloud	41.1%
MiniMax M2.7	#9	MiniMax	39.8%
Qwen3.6 27B	#10	Alibaba Cloud	36.2%
Ornith-1.0-35B	#11	DeepReinforce AI	34.6%
Qwen3.6-35B-A3B	#12	Alibaba Cloud	29.4%
Ornith-1.0-9B	#13	DeepReinforce AI	27.2%

Benchmark score table (13 models)

One best score per model · higher is better

NL2Repo leaderboard ranked by score
Rank	Model	Provider	License	Evidence use	Score
#1	GLM-5.2 glm-5-2	Z.AI	open	Estimated reference	48.9%
#2	Ornith-1.0-397B ornith-1-0-397b	DeepReinforce AI	open	Estimated reference	48.2%
#3	Qwen3.7-Max qwen3.7-max	Alibaba Cloud	—	Reference only	47.2%
#4	Claude Opus 4.5 claude-opus-4-5	Anthropic	closed	Reference only	43.2%
#5	Qwen 3.6 Max (preview) qwen3-6-max-preview	Alibaba Cloud	closed	Reference only	42.9%
#6	GLM-5.1 glm-5-1	Z.AI	open	Reference only	42.7%
#7	MiniMax M3 MiniMax-M3	MiniMax	—	Reference only	42.1%
#8	Qwen3.7-Plus qwen3.7-plus	Alibaba Cloud	—	Reference only	41.1%
#9	MiniMax M2.7 minimax-m2-7	MiniMax	open	Reference only	39.8%
#10	Qwen3.6 27B qwen3.6-27b	Alibaba Cloud	open	Estimated reference	36.2%
#11	Ornith-1.0-35B ornith-1-0-35b	DeepReinforce AI	open	Estimated reference	34.6%
#12	Qwen3.6-35B-A3B qwen3-6-35b-a3b	Alibaba Cloud	open	Estimated reference	29.4%
#13	Ornith-1.0-9B ornith-1-0-9b	DeepReinforce AI	open	Estimated reference	27.2%

The top of this snapshot is led by GLM-5.2 at 48.9%; third place is 1.7 points behind. The top-10 spread is 12.7 points.

Compare top models

GLM-5.2 vs Ornith-1.0-397B Ornith-1.0-397B vs Qwen3.7-Max Qwen3.7-Max vs Claude Opus 4.5 Claude Opus 4.5 vs Qwen 3.6 Max (preview)

About NL2Repo

Definition and scoring

Organisation: MiniMax
Category: Coding
Version: 2026
Direction: higher is better
Ranking use: Reference
Contamination risk: Unknown

A repository-understanding benchmark that measures whether models can map natural-language requests onto the right code locations and system changes. Results stay tied to the exact model variant and evaluation system. Multiple systems for the same model use the best published score on this page; overall Lumina scoring uses the median of ranking-eligible rows.

Open benchmark source ↗

FAQ

Quick answers

What does NL2Repo measure?

A repository-understanding benchmark that measures whether models can map natural-language requests onto the right code locations and system changes.

Which model scores highest on NL2Repo?

GLM-5.2 by Z.AI currently leads with 48.9%.

How many models are evaluated on NL2Repo?

13 models in the LuminaBench cohort have a qualifying score on this benchmark.

Does this affect overall Lumina rank?

No. This benchmark is display-only and does not enter the overall Lumina composite.

LiveCodeBench

Rolling · 44 results · Weighted 3.2%

SWE-bench Verified

Verified · 79 results · Weighted 3.2%

CRUXEval

Rolling · Not evaluated · Not Weighted

SciCode

Rolling · 238 results · Weighted 3.2%

BigCodeBench

Rolling · 2 results · Not Weighted

SWE-Bench Pro

v1 · 92 results · Weighted 3.2%

Coding · Benchmark profile

NL2Repo

A repository-understanding benchmark that measures whether models can map natural-language requests onto the right code locations and system changes.

Reference2026Active13 models

Data verified 21 Jul 2026 · Methodology 1.6.0

Benchmark score on NL2Repo

Leaderboard · 13 models

GLM-5.2 leads at 48.9%, followed by Ornith-1.0-397B (48.2%) and Qwen3.7-Max (47.2%).

Display onlyCoding2026

#1Open

GLM-5.2

Z.AI

glm-5-2

48.9%

Overall #25 · Context 1M tokens

#2Open

Ornith-1.0-397B

DeepReinforce AI

ornith-1-0-397b

48.2%

Overall — · Context 256K tokens

#3Closed

Qwen3.7-Max

Alibaba Cloud

qwen3.7-max

47.2%

Overall #37 · Context 1M tokens

Visual analysis

See the field.

Switch between model placement, score distribution and descriptive provider averages. Every view uses the same sourced leaderboard.

ViewModelsColours

Z.AIDeepReinforce AIAlibaba CloudAnthropicMiniMax

NL2Repo leaderboard

13 ranked models · higher is better

View accessible chart data

Accessible data for NL2Repo leaderboard
Model	Rank	Provider	Score
GLM-5.2	#1	Z.AI	48.9%
Ornith-1.0-397B	#2	DeepReinforce AI	48.2%
Qwen3.7-Max	#3	Alibaba Cloud	47.2%
Claude Opus 4.5	#4	Anthropic	43.2%
Qwen 3.6 Max (preview)	#5	Alibaba Cloud	42.9%
GLM-5.1	#6	Z.AI	42.7%
MiniMax M3	#7	MiniMax	42.1%
Qwen3.7-Plus	#8	Alibaba Cloud	41.1%
MiniMax M2.7	#9	MiniMax	39.8%
Qwen3.6 27B	#10	Alibaba Cloud	36.2%
Ornith-1.0-35B	#11	DeepReinforce AI	34.6%
Qwen3.6-35B-A3B	#12	Alibaba Cloud	29.4%
Ornith-1.0-9B	#13	DeepReinforce AI	27.2%

Benchmark score table (13 models)

One best score per model · higher is better

NL2Repo leaderboard ranked by score
Rank	Model	Provider	License	Evidence use	Score
#1	GLM-5.2 glm-5-2	Z.AI	open	Estimated reference	48.9%
#2	Ornith-1.0-397B ornith-1-0-397b	DeepReinforce AI	open	Estimated reference	48.2%
#3	Qwen3.7-Max qwen3.7-max	Alibaba Cloud	—	Reference only	47.2%
#4	Claude Opus 4.5 claude-opus-4-5	Anthropic	closed	Reference only	43.2%
#5	Qwen 3.6 Max (preview) qwen3-6-max-preview	Alibaba Cloud	closed	Reference only	42.9%
#6	GLM-5.1 glm-5-1	Z.AI	open	Reference only	42.7%
#7	MiniMax M3 MiniMax-M3	MiniMax	—	Reference only	42.1%
#8	Qwen3.7-Plus qwen3.7-plus	Alibaba Cloud	—	Reference only	41.1%
#9	MiniMax M2.7 minimax-m2-7	MiniMax	open	Reference only	39.8%
#10	Qwen3.6 27B qwen3.6-27b	Alibaba Cloud	open	Estimated reference	36.2%
#11	Ornith-1.0-35B ornith-1-0-35b	DeepReinforce AI	open	Estimated reference	34.6%
#12	Qwen3.6-35B-A3B qwen3-6-35b-a3b	Alibaba Cloud	open	Estimated reference	29.4%
#13	Ornith-1.0-9B ornith-1-0-9b	DeepReinforce AI	open	Estimated reference	27.2%

The top of this snapshot is led by GLM-5.2 at 48.9%; third place is 1.7 points behind. The top-10 spread is 12.7 points.

Compare top models

GLM-5.2 vs Ornith-1.0-397B Ornith-1.0-397B vs Qwen3.7-Max Qwen3.7-Max vs Claude Opus 4.5 Claude Opus 4.5 vs Qwen 3.6 Max (preview)

About NL2Repo

Definition and scoring

Organisation: MiniMax
Category: Coding
Version: 2026
Direction: higher is better
Ranking use: Reference
Contamination risk: Unknown

Open benchmark source ↗

FAQ

Quick answers

What does NL2Repo measure?

A repository-understanding benchmark that measures whether models can map natural-language requests onto the right code locations and system changes.

Which model scores highest on NL2Repo?

GLM-5.2 by Z.AI currently leads with 48.9%.

How many models are evaluated on NL2Repo?

13 models in the LuminaBench cohort have a qualifying score on this benchmark.

Does this affect overall Lumina rank?

No. This benchmark is display-only and does not enter the overall Lumina composite.

LiveCodeBench

Rolling · 44 results · Weighted 3.2%

SWE-bench Verified

Verified · 79 results · Weighted 3.2%

CRUXEval

Rolling · Not evaluated · Not Weighted

SciCode

Rolling · 238 results · Weighted 3.2%

BigCodeBench

Rolling · 2 results · Not Weighted

SWE-Bench Pro

v1 · 92 results · Weighted 3.2%

Search LUMINA BENCH

Navigation

NL2Repo

Leaderboard · 13 models

GLM-5.2

Ornith-1.0-397B

Qwen3.7-Max

See the field.

NL2Repo leaderboard

Crop NL2Repo leaderboard

Benchmark score table (13 models)

Crop NL2Repo benchmark table

Compare top models

Definition and scoring

Quick answers

What does NL2Repo measure?

Which model scores highest on NL2Repo?

How many models are evaluated on NL2Repo?

Does this affect overall Lumina rank?

NL2Repo

Leaderboard · 13 models

GLM-5.2

Ornith-1.0-397B

Qwen3.7-Max

See the field.

NL2Repo leaderboard

Crop NL2Repo leaderboard

Benchmark score table (13 models)

Crop NL2Repo benchmark table

Compare top models

Definition and scoring

Quick answers

What does NL2Repo measure?

Which model scores highest on NL2Repo?

How many models are evaluated on NL2Repo?

Does this affect overall Lumina rank?

Leaderboard · 13 models

GLM-5.2

Ornith-1.0-397B

Qwen3.7-Max

See the field.

Benchmark score table (13 models)

Compare top models

Definition and scoring

Quick answers

What does NL2Repo measure?

Which model scores highest on NL2Repo?

How many models are evaluated on NL2Repo?

Does this affect overall Lumina rank?

More Coding benchmarks

Leaderboard · 13 models

GLM-5.2

Ornith-1.0-397B

Qwen3.7-Max

See the field.

Benchmark score table (13 models)

Compare top models

Definition and scoring

Quick answers

What does NL2Repo measure?

Which model scores highest on NL2Repo?

How many models are evaluated on NL2Repo?

Does this affect overall Lumina rank?

More Coding benchmarks