Named Entity Recognition at Scale:...

Entity Recognition Beyond the Tutorial

Named Entity Recognition on benchmark datasets is a solved problem. NER on your company's actual data — with its domain jargon, inconsistent formatting, and entities that don't appear in any training set — is very much not solved. The gap between 95% F1 on CoNLL-2003 and useful entity extraction from production documents is where most NER projects die quietly.

I've spent the last three years building NER systems for legal documents, financial reports, and biomedical literature. Each domain broke standard approaches in different ways, and each required domain-specific engineering that no pre-trained model could provide out of the box.

Why Standard NER Fails on Domain Text

Pre-trained NER models recognize person names, organizations, locations, and dates. That covers maybe 30% of what enterprise users actually need to extract. The rest is domain-specific: drug names, legal citations, financial instrument identifiers, product codes, gene symbols. These entities follow patterns that generic models haven't seen during training.

The first instinct is to fine-tune. The problem is annotation cost. Getting 5,000 annotated examples for a new entity type takes weeks of domain expert time at minimum. If you need to add 20 entity types for a new vertical, you're looking at months of annotation before you can even start training. That timeline doesn't work for most product teams.

We covered a related topic in Sentence Embedding Models: Contrastive Learning and Evaluati.

A Hybrid Architecture

The system that actually works combines three approaches at different precision-recall tradeoffs:

class HybridNER:
 def __init__(self):
 self.neural_ner = None
 self.pattern_ner = None
 self.few_shot_ner = None

 def extract(self, text):
 pattern_entities = self.pattern_ner.extract(text)
 neural_entities = self.neural_ner.extract(text)
 few_shot_entities = self.few_shot_ner.extract(text)
 merged = self._resolve_conflicts(
 pattern_entities, neural_entities, few_shot_entities
 )
 return merged

 def _resolve_conflicts(self, *entity_lists):
 all_entities = []
 for priority, entities in enumerate(entity_lists):
 for ent in entities:
 ent['priority'] = priority
 all_entities.append(ent)
 all_entities.sort(key=lambda e: (e['start'], -e['priority']))
 resolved = []
 last_end = -1
 for ent in all_entities:
 if ent['start'] >= last_end:
 resolved.append(ent)
 last_end = ent['end']
 return resolved

Pattern-Based Extraction for Structured Entities

Don't underestimate regex. For entities with consistent formatting — ICD codes, CAS numbers, patent identifiers, IP addresses — a well-crafted pattern extracts with near-perfect precision. The trick is building a maintainable pattern library rather than a pile of one-off regular expressions that nobody can read six months later.

ENTITY_PATTERNS = {
 'icd10_code': {
 'pattern': r'\b[A-Z]\d{2}(?:\.\d{1,4})?\b',
 'validator': lambda m: m.group()[0] in 'ABCDEFGHIJKLMNOPQRSTUVYZ',
 'context_window': 50,
 },
 'cas_number': {
 'pattern': r'\b\d{2,7}-\d{2}-\d\b',
 'validator': lambda m: validate_cas_checksum(m.group()),
 },
 'doi': {
 'pattern': r'10\.\d{4,9}/[-._;()/:A-Z0-9]+',
 'validator': None,
 },
}

The validator functions are critical. A regex for CAS numbers will match plenty of strings that look like CAS numbers but aren't. The checksum validation catches these false positives cheaply. We've also found that adding a context window — checking the text around a match for confirming keywords — reduces false positives by another 15-20% for ambiguous patterns.

Few-Shot NER for Rapid Entity Type Addition

When a new entity type shows up and you don't have time for annotation campaigns, few-shot NER is the fallback. You provide 10-20 examples of the entity, and the system learns to recognize similar spans using embedding similarity.

This won't match the precision of a fine-tuned model, but it gets you from zero to useful in hours instead of months. We treat it as a bootstrapping tool — the few-shot model's predictions become annotation suggestions, humans correct them, and once you've accumulated enough labeled data, you promote the entity type to the neural model. This active learning loop is how we've scaled from 5 entity types to 47 over eighteen months without ever running a dedicated annotation campaign.

Scaling to Millions of Documents

At our document volumes (2M+ documents per day), NER can't be a synchronous process. We run extraction as a background pipeline with three stages: fast pattern matching eliminates documents that clearly don't contain entities of interest (this filters out about 60% of incoming documents), neural NER processes the remaining documents in GPU-accelerated batches, and results get written to an entity index for downstream search and analytics.

Batching for the neural model is where most of the throughput engineering lives. Dynamic batching by sequence length — grouping similar-length documents together — improves GPU utilization from roughly 40% to 85% on our hardware. The sorting overhead is trivial compared to the GPU time saved from not padding short sequences to the length of the longest document in the batch.

We also learned the hard way that GPU batch size and throughput aren't the only bottleneck. Our initial implementation spent more time deserializing documents from the message queue than running inference. Switching from JSON to Protocol Buffers for the internal message format cut preprocessing time by 70% and nearly doubled end-to-end throughput without touching the model at all.

NER at production scale is fundamentally an engineering problem layered on top of a modeling problem. Get the engineering wrong, and even the best model won't deliver results at the speed and cost your business requires. Get it right, and a decent model outperforms a great model stuck behind inefficient infrastructure.

Named Entity Recognition at Scale: Handling Domain-Specific Entities