NER OSS Reference Benchmark

Summary

English: strongest average flat F1 is Flair NER EN at 0.720; fastest average p50 is spaCy EntityRecognizer XX at 1.54 ms.
Polish: strongest average flat F1 is Stanza NER PL at 0.684; fastest average p50 is spaCy EntityRecognizer XX at 6.31 ms.
Caveat: 21 long documents were truncated to 384 tokens (input length range 1217-2645).

Run Metadata

Field	Value
Benchmark	NER OSS Reference Benchmark
Generated at	2026-06-05T14:33:28.697085+00:00
Processor	Apple M1 Pro
Python	3.12.10

English Quality And Speed

Dataset	Best quality	F1	p50 ms	Fastest	p50 ms	Fastest F1
CoNLL-2003 EN Test	Flair NER EN	0.912	42.44	spaCy EntityRecognizer XX	1.38	0.604
WikiNER EN Test	spaCy EntityRecognizer XX	0.811	2.05	spaCy EntityRecognizer XX	2.05	0.811
WikiANN EN Test	Flair NER EN	0.491	34.81	spaCy EntityRecognizer XX	1.21	0.447

Polish Quality And Speed

Dataset	Best quality	F1	p50 ms	Fastest	p50 ms	Fastest F1
PolEval-2018 PL Test	Stanza NER PL	0.732	581.55	spaCy EntityRecognizer XX	16.01	0.372
WikiNER PL Test	spaCy EntityRecognizer XX	0.857	1.68	spaCy EntityRecognizer XX	1.68	0.857
WikiANN PL Test	Stanza NER PL	0.656	43.89	spaCy EntityRecognizer XX	1.25	0.610

English Model x Dataset Heatmaps

Higher F1 is strongerHigher p50 is slower

Flat F1

Provider	CoNLL-2003 EN Test	WikiNER EN Test	WikiANN EN Test
Flair NER EN	0.912	0.757	0.491
Flair NER EN OntoNotes	0.546	0.541	0.366
GLiNER Multi	0.357	0.351	0.269
Stanza NER EN	0.495	0.537	0.379
spaCy EntityRecognizer EN	0.438	0.410	0.311
spaCy EntityRecognizer XX	0.604	0.811	0.447

p50 Latency

Provider	CoNLL-2003 EN Test	WikiNER EN Test	WikiANN EN Test
Flair NER EN	42.44	108.38	34.81
Flair NER EN OntoNotes	42.74	107.97	35.19
GLiNER Multi	59.96	68.63	59.96
Stanza NER EN	46.86	117.37	39.84
spaCy EntityRecognizer EN	3.38	5.30	2.87
spaCy EntityRecognizer XX	1.38	2.05	1.21

Polish Model x Dataset Heatmaps

Higher F1 is strongerHigher p50 is slower

Flat F1

Provider	PolEval-2018 PL Test	WikiNER PL Test	WikiANN PL Test
GLiNER Multi	0.166	0.524	0.426
Stanza NER PL	0.732	0.666	0.656
spaCy EntityRecognizer PL	0.632	0.527	0.463
spaCy EntityRecognizer XX	0.372	0.857	0.610

p50 Latency

Provider	PolEval-2018 PL Test	WikiNER PL Test	WikiANN PL Test
GLiNER Multi	307.28	68.18	60.58
Stanza NER PL	581.55	104.34	43.89
spaCy EntityRecognizer PL	61.40	6.38	4.66
spaCy EntityRecognizer XX	16.01	1.68	1.25

English Leaderboard

Provider	Official benchmark	Official F1	CoNLL-2003 EN Test	WikiNER EN Test	WikiANN EN Test
spaCy EntityRecognizer EN	OntoNotes 5	0.843	0.438	0.410	0.311
spaCy EntityRecognizer XX	WikiNER	0.831	0.604	0.811	0.447
Flair NER EN	CoNLL-03	0.929	0.912	0.757	0.491
Flair NER EN OntoNotes	OntoNotes	0.893	0.546	0.541	0.366
Stanza NER EN	OntoNotes	0.888	0.495	0.537	0.379
GLiNER Multi	n/a	n/a	0.357	0.351	0.269

Polish Leaderboard

Provider	Official benchmark	Official F1	PolEval-2018 PL Test	WikiNER PL Test	WikiANN PL Test
spaCy EntityRecognizer PL	NKJP / package validation	0.804	0.632	0.527	0.463
spaCy EntityRecognizer XX	WikiNER	0.831	0.372	0.857	0.610
Stanza NER PL	NKJP	0.887	0.732	0.666	0.656
GLiNER Multi	n/a	n/a	0.166	0.524	0.426

Artifacts

Artifact	Link
Benchmark brief	HTML / Markdown
Full report	HTML / Markdown
Capability matrix	HTML / Markdown
Run summary	JSON
CSV exports	dataset_metrics.csv