NER OSS Reference Benchmark Brief

Generated at: 2026-06-05T14:33:28.693573+00:00

This file is the communication-first companion to benchmark_report.md.

Machine

Processor: Apple M1 Pro
Logical CPUs: 8
Python: 3.12.10

Headline Findings

English: strongest average flat F1 is Flair NER EN at 0.720; fastest average p50 is spaCy EntityRecognizer XX at 1.54 ms.
Polish: strongest average flat F1 is Stanza NER PL at 0.684; fastest average p50 is spaCy EntityRecognizer XX at 6.31 ms.
Caveat: 21 long documents were truncated to 384 tokens (input length range 1217-2645).

English Quality And Speed Snapshot

Dataset	Best quality	F1	p50 ms	Fastest	p50 ms	Fastest F1
CoNLL-2003 EN Test	Flair NER EN	0.912	42.44	spaCy EntityRecognizer XX	1.38	0.604
WikiNER EN Test	spaCy EntityRecognizer XX	0.811	2.05	spaCy EntityRecognizer XX	2.05	0.811
WikiANN EN Test	Flair NER EN	0.491	34.81	spaCy EntityRecognizer XX	1.21	0.447

Polish Quality And Speed Snapshot

Dataset	Best quality	F1	p50 ms	Fastest	p50 ms	Fastest F1
PolEval-2018 PL Test	Stanza NER PL	0.732	581.55	spaCy EntityRecognizer XX	16.01	0.372
WikiNER PL Test	spaCy EntityRecognizer XX	0.857	1.68	spaCy EntityRecognizer XX	1.68	0.857
WikiANN PL Test	Stanza NER PL	0.656	43.89	spaCy EntityRecognizer XX	1.25	0.610

Reading Guide

F1 is the main quality score for span + label correctness on the current dataset.
p50 ms is median per-document latency on the machine above.
Best quality answers who wins on quality; Fastest answers who returns first.
Full matched / predicted / gold / precision / recall / p95 details stay in benchmark_report.md.