Named Entity Recognition: SpaCy, Transformers and Zero-Shot Approaches

Named Entity Recognition remains one of the most practical and frequently deployed tasks in natural language processing, yet the gap between a demo that tags "Apple" as an ORG and a production system that reliably extracts domain-specific entities from messy text is substantial. In my experience building NER systems for legal contracts, biomedical literature, and customer support transcripts, the choice between SpaCy, fine-tuned transformers, and zero-shot LLM approaches is rarely about raw accuracy alone. It is about annotation budget, latency constraints, maintainability, and how your entity definitions will evolve over time. This article walks through all three approaches from an engineering perspective — how they work under the hood, when to use each one, how to implement them correctly, and what tradeoffs you will face when you move from notebook to production.

The Architectural Divide: Statistical NER in SpaCy vs. Transformer Token Classification

At a conceptual level, SpaCy and transformer-based NER solve the same problem — assign a label like PERSON, ORG, GPE, or a custom type to spans of text — but their architectures lead to very different behaviors, failure modes, and tuning strategies. Understanding this divide helps you predict where each system will struggle.

How SpaCy's Transition-Based Parser Handles Entities

SpaCy's default NER component is not a simple per-token classifier. It uses a transition-based system with a shallow convolutional neural network (CNN) and Bloom embeddings to predict actions that construct entity spans. The model looks at a token in context, predicts whether to Begin, Inside, Outside, or mark the end of an entity, and moves through the document incrementally. This makes it fast and non-autoregressive at the token level but sensitive to local context and tokenization decisions.

I've found that this architecture is excellent for high-throughput scenarios and for entities with relatively consistent lexical and orthographic signals. For example, it learns quickly that capitalized sequences followed by "Inc." or "LLC" are likely ORGs, or that patterns like "Dr. [A-Z][a-z]+" are PERSON mentions. Where it struggles is with long-range dependencies and ambiguous entities that require world knowledge. If your text contains "Washington was unanimous in his decision" versus "Washington is moving to increase tariffs," the CNN has a much narrower effective context window than a transformer to disambiguate PERSON from GPE. The model also relies heavily on the hash-based Bloom embeddings for words, prefixes, suffixes, and shapes, which makes it memory-efficient but less capable of capturing deep semantic similarity between rare words.

Why Transformers Reframe NER as Token Classification

Transformer models like BERT, RoBERTa, and DeBERTa reframe NER as a token classification problem over subword tokens. Each token gets a contextual embedding from the self-attention stack, then a linear layer predicts a label in the BIO or BILUO scheme. Because every token attends to every other token in the sequence, the model can resolve ambiguity using the full sentence or even paragraph context.

The practical consequence is significant. A fine-tuned RoBERTa will correctly label "Jordan" as PERSON in "Jordan played exceptionally well" and as GPE in "The treaty was signed in Jordan" far more reliably than a CNN, because the contextualized representation for "Jordan" is entirely different in those two sentences. The cost is that you are now running a 110M to 350M parameter model with quadratic attention complexity. You also inherit subword tokenization headaches: a word like "BioNTech" might be split into "Bio", "N", "Tech", and you must align your word-level labels to subwords, decide whether to label only the first subword or all of them, and handle decoding back to entity spans without creating invalid tag sequences like I-PER following O.

Both families output BIO/BILUO tags, but I've seen teams get tripped up by the decoding step. You need a post-processing layer that converts token-level predictions to span-level entities, handles subword aggregation, and filters out illegal transitions. The Hugging Face token classification pipeline does this by default, but when you write a custom inference loop using PyTorch Documentation, you must implement it yourself.

Training and Customizing SpaCy's NER Pipeline for Domain-Specific Entities

SpaCy shines when you need to add custom entity types quickly without massive compute. I recently used it to extract CLAUSE_TYPE, PARTY, and EFFECTIVE_DATE from a corpus of 8,000 service agreements. The base en_core_web_trf model already knew PERSON and DATE, but it had no concept of a contractual clause. Training a custom NER component on top of SpaCy's pipeline was far cheaper than labeling 50k examples for a transformer.

The workflow that has worked most reliably for me starts with creating a blank or base model, adding an NER pipe, and training with a mix of your new examples and a small portion of the original training data to prevent catastrophic forgetting. If you overwrite the existing NER entirely, you will lose its ability to recognize generic entities, which often serve as useful features for your downstream tasks.

Annotation Strategy and Data Preparation for SpaCy

For SpaCy, annotation quality matters more than volume. I aim for 1,500-3,000 high-quality sentences per custom entity type, with careful attention to boundary consistency. One of the most common errors I see is inconsistent span boundaries — for example, labeling "Advanced Micro Devices, Inc." in one sentence but just "Advanced Micro Devices" in another. SpaCy's transition system is very sensitive to this, and it will learn to be uncertain about trailing punctuation and legal suffixes.

Use BILUO encoding internally, as SpaCy does by default, because it explicitly marks single-token entities (U) and last tokens (L), which gives slightly more learning signal than BIO. When you prepare your data with DocBin, make sure your character spans align with token boundaries. SpaCy will silently discard misaligned spans during training, and I have debugged projects where 12% of the annotations were being dropped for this reason.

# Training a custom NER component in SpaCy 3.x
import spacy
from spacy.tokens import DocBin
from spacy.training import Example

# Load a base model to retain general NER knowledge
nlp = spacy.load("en_core_web_sm")

# Create a new NER pipe or get the existing one
if "ner" not in nlp.pipe_names:
    ner = nlp.add_pipe("ner", last=True)
else:
    ner = nlp.get_pipe("ner")

# Add new labels - critical step before training
ner.add_label("CLAUSE_TYPE")
ner.add_label("PARTY")

# Example data preparation - ensure spans are token-aligned
# Use offsets and let SpaCy handle BILUO conversion
TRAIN_DATA = [
    ("The Indemnification clause survives termination by Acme Corp.", 
     {"entities": [(4, 19, "CLAUSE_TYPE"), (43, 52, "PARTY")]}),
]

# Training loop with compounding batch sizes
optimizer = nlp.resume_training()
for epoch in range(30):
    for text, annotations in TRAIN_DATA:
        doc = nlp.make_doc(text)
        example = Example.from_dict(doc, annotations)
        nlp.update([example], sgd=optimizer)

Practical Tuning Tips I Use for SpaCy

First, do not train the entire pipeline if you only need NER. Freeze the tagger and parser if you are not updating them, which stabilizes training and halves iteration time. Second, use spacy train via the config system rather than the manual loop above for any real project — the config handles batching, learning rate schedules, early stopping, and multi-GPU correctly. Third, add a small amount of negative examples: sentences that contain no entities at all. Without them, the model learns a prior that every sentence should contain an entity and precision drops. Finally, evaluate with span-level F1, not token-level accuracy. A prediction that gets the boundaries slightly wrong is a complete miss in most applications, and token accuracy will hide that.

If your task is primarily classification of already-identified spans rather than discovery, consider whether a Text Classification Pipelines for Production: From Training to Serving approach combined with rule-based candidate generation would be simpler than full NER.

Fine-Tuning BERT and RoBERTa for High-Accuracy Entity Extraction

When accuracy is paramount and you have at least a few thousand labeled sentences, fine-tuning a transformer encoder is still the most reliable path to state-of-the-art results on standard entity types and many domain-specific ones. In my benchmarks on financial filings, a fine-tuned roberta-base consistently outperformed SpaCy's CNN by 6-9 points of span F1, particularly on ambiguous ORG vs. PRODUCT distinctions.

The implementation is conceptually straightforward but has several engineering traps. You are not just calling Trainer.train() and walking away. You need to handle label alignment, class imbalance, long documents, and decoding.

Label Alignment, Loss Masking, and Handling Long Sequences

Transformers use WordPiece or BPE tokenizers, so one word can become multiple subwords. The standard practice, as documented in the Hugging Face Documentation, is to assign the word's label to its first subword and set the loss label for subsequent subwords to -100 so they are ignored in the cross-entropy calculation. This prevents the model from being penalized multiple times for a single word and avoids teaching it to predict B-PER on a continuation subword like "##son" in "Johnson".

For documents longer than 512 tokens, you must use a sliding window with stride. I typically use a window of 384 tokens with a stride of 128, then aggregate predictions by averaging logits in overlapping regions or by taking the prediction from the window where the token is most central. Without this, you will truncate the tail of your documents and miss entities at the end, which is a common silent failure in production.

Class imbalance is another real issue. In most corpora, O tags dominate 85-90% of tokens. If you use plain cross-entropy, the model can achieve high accuracy by predicting O everywhere. I often add class weights or use a focal loss variant, and I monitor per-entity-type recall closely during training.

from transformers import AutoTokenizer, AutoModelForTokenClassification
from transformers import Trainer, TrainingArguments
import torch

model_name = "roberta-base"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForTokenClassification.from_pretrained(
    model_name, num_labels=len(label_list)
)

def tokenize_and_align_labels(examples):
    tokenized = tokenizer(
        examples["tokens"], 
        is_split_into_words=True, 
        truncation=True, 
        max_length=512
    )
    labels = []
    for i, word_labels in enumerate(examples["ner_tags"]):
        word_ids = tokenized.word_ids(batch_index=i)
        previous_word_idx = None
        label_ids = []
        for word_idx in word_ids:
            if word_idx is None:
                label_ids.append(-100)
            elif word_idx != previous_word_idx:
                label_ids.append(word_labels[word_idx])
            else:
                # Subsequent subwords get ignored in loss
                label_ids.append(-100)
            previous_word_idx = word_idx
        labels.append(label_ids)
    tokenized["labels"] = labels
    return tokenized

# Apply to dataset and train with weighted loss if needed
# See PyTorch docs for custom Trainer.compute_loss with class weights

Choosing the Right Encoder and Decoding Strategy

For English NER, roberta-base or deberta-v3-base usually gives the best accuracy-to-speed ratio. If you need multilingual support, xlm-roberta-base is a strong default. I avoid using large models unless the F1 gain justifies the 2-3x inference cost — in my tests, roberta-large added only 1.2 F1 on CoNLL-03 but doubled latency.

After fine-tuning, decode with care. Raw token predictions often produce invalid sequences like B-ORG followed by I-PER. I apply a simple rule-based correction or a CRF layer on top of the encoder. A CRF adds minimal latency and eliminates illegal transitions while improving span F1 by about 0.5-1 point. For evaluation, use seqeval's strict span-level F1, which requires exact boundary and type match, rather than the more forgiving token-level metrics.

Zero-Shot NER with Instruction-Tuned LLMs: When Labels Are Cheap and Data Is Scarce

Zero-shot and few-shot NER with large language models has changed how I approach new entity extraction problems where annotation budget is zero or the label schema is still evolving. Instead of training a classifier, you prompt an instruction-tuned model to extract entities and return them in a structured format. In my experience, this works surprisingly well for exploratory work and for rare, semantically complex entities that are hard to define with examples alone.

The key limitation is reliability. LLMs are excellent at understanding what you mean by "ACQUISITION_TARGET" but inconsistent at adhering to exact span boundaries, handling large documents, and producing predictable JSON. They also hallucinate entities that are not verbatim in the source text, which is unacceptable in many compliance-sensitive use cases.

Prompt Design That Actually Works for Zero-Shot Extraction

I have iterated through many prompt templates, and the pattern that yields the highest precision is: provide crisp, one-sentence definitions for each entity type, explicitly state what NOT to extract, require verbatim copying, and give two or three few-shot examples with correct outputs. Without the negative instructions, models tend to over-extract. For example, if you define ORG broadly, the model may label "the board" or "management" as ORG.

For structured output, I instruct the model to return a JSON list of objects with "text" and "label" keys, then validate that every "text" value is a substring of the input. Any entry that fails this check is discarded. This simple validation catches about 90% of hallucinations. For longer documents, chunk into overlapping 1,500-character windows and deduplicate results, because asking an LLM to extract from a 10k-token contract in one call leads to recall drops in the middle of the document.

This paradigm connects directly to modern prompting methods. The selection of examples, the use of chain-of-thought to have the model justify its extraction before outputting JSON, and the iterative refinement of label definitions all benefit from a disciplined approach. I recommend reading Systematic Prompt Engineering: Chain-of-Thought, Few-Shot and Tool Use if you plan to systematize your prompt development rather than hand-tuning.

import json

# Zero-shot NER via instruction-tuned LLM - validated extraction
prompt = """
You are an entity extractor. Extract only verbatim spans from the input text.

Entity definitions:
- PERSON: Real person's full name (e.g., "Marie Curie")
- ORG: Company or institution name (e.g., "Pfizer Inc.")
- PRODUCT: Drug or device name (e.g., "mRNA-1273")

Rules:
- Copy text exactly as it appears. Do not paraphrase or normalize.
- Return JSON only: [{"text": "...", "label": "...", "start": int}]
- If no entities, return []

Text: "Dr. Sarah Chen from BioNTech presented data on BNT162b2 at the FDA meeting."

Examples:
Input: "Tim Cook announced the iPhone at Apple Park."
Output: [{"text": "Tim Cook", "label": "PERSON", "start": 0}, {"text": "Apple Park", "label": "ORG", "start": 32}]
"""

# Pseudocode for validation loop
# response = llm.generate(prompt)
# entities = json.loads(response)
# validated = [e for e in entities if e["text"] in source_text]
# Offsets via source_text.find(e["text"]) with handling for duplicates

When Zero-Shot Is the Wrong Tool

I've learned to avoid zero-shot LLM extraction in three situations: first, when latency or cost matters — an API call that takes 1.5 seconds and costs $0.01 per document does not scale to millions of documents; second, when you need exact character offsets for downstream highlighting or relation extraction — LLM offsets are frequently off by one or two characters; and third, when you have a stable schema and enough data to fine-tune. In those cases, distill the LLM's extractions into a training set and fine-tune a smaller model. I often use an LLM to label 10k unlabeled sentences, manually correct 1k of them, then train a SpaCy or transformer model. This semi-supervised loop gives you 80% of the LLM quality at 5% of the inference cost.

Also consider that embeddings and retrieval can complement zero-shot NER. If your entity definitions depend heavily on context from a knowledge base — for example, distinguishing a product name from a generic term — retrieving relevant descriptions first can improve extraction. See Text Embedding Models: Sentence-BERT, E5 and Cohere Embed Compared for a practical comparison of embedding options.

Deploying NER in Production: Latency, Memory, and Pipeline Integration Tradeoffs

Moving NER to production surfaces a different set of concerns than optimizing F1 on a held-out test set. In production, you care about p99 latency, memory footprint, model versioning, and how the component interacts with the rest of your pipeline. I've seen technically excellent transformer models get rejected by platform teams because they required a GPU for every inference pod, while a SpaCy pipeline that was 2 points lower in F1 shipped the next week.

Latency and Throughput Strategies

For CPU-only services, SpaCy is hard to beat. A quantized SpaCy pipeline processes ~800-1,200 sentences per second on a single 4-core instance. A transformer fine-tuned on roberta-base, even after ONNX conversion and quantization to INT8, typically handles 60-150 sentences per second on the same hardware. If you must use a transformer on CPU, batch requests aggressively, use dynamic batching, and compile to ONNX Runtime with graph optimizations. On GPU, throughput improves dramatically — I regularly see 1,500+ sentences per second on a T4 with batch size 32 — but you add operational complexity and cost.

For zero-shot LLM approaches, the bottleneck is network and generation. Even with streaming, median latency is 800ms to 2 seconds per document. Caching helps if you see repeated texts, but for novel user-generated content, it rarely hits. My rule of thumb: if your service-level objective is under 200ms per request, do not use a remote LLM for NER on the critical path. Use it offline for data labeling or for a separate async enrichment path, and serve a distilled model synchronously.

Integration, Monitoring, and Schema Evolution

NER rarely lives alone. Its outputs feed entity linking, relation extraction, or a retrieval-augmented generation system where extracted entities become filters or query expansions. When designing for RAG Architecture: Building Retrieval-Augmented Generation Systems, consider whether your retriever needs span-level provenance. If so, ensure your NER component returns character offsets aligned to the original text, not to normalized or lowercased copies.

Monitoring is critical because entity distributions drift. A model trained on 2022 news will underperform on new company names and products that emerged in 2024. I log the rate of each entity type per day, track the out-of-vocabulary entity ratio, and set alerts when the model's confidence distribution shifts significantly. For transformer models, I also monitor the fraction of tokens predicted as O — a sudden increase often signals a tokenization or preprocessing regression. For SpaCy, I re-evaluate on a small golden set of 200 manually labeled examples weekly; for LLMs, I sample 50 extractions for human review per week because automated metrics are less trustworthy.

Finally, plan for schema evolution. Business stakeholders will ask for a new entity type six weeks after launch. With SpaCy and transformers, that means annotation and retraining. With zero-shot LLMs, it means prompt updates. I now build my NER service with a pluggable extraction layer so I can route certain entity types to a transformer and experimental types to an LLM, then gradually migrate as definitions stabilize. This hybrid architecture has proven the most maintainable across multiple projects.

Frequently Asked Questions