NER: general vs fine-tuned, and when to switch
Start with a general NER model to learn what you need. Fine-tune a small model once your entities are specific to your domain, and handle negation and context separately.
When to fine-tune a small model instead of prompting a large one, how to measure the difference, and what we have shipped.
Fine-tuning means training a model further on examples of your own task, so that it does one job well. For narrow, high-volume work it can beat prompting a large model on cost, speed and privacy. It only pays off if you can prove it, which is why most of what we write here is about measurement.
These notes cover when a small fine-tuned model is the right tool, how to build the held-out test set that shows whether it worked, and what to keep outside the model, such as rules for negation and history. Our examples come mostly from clinical text, where the entities are specific, the data is sensitive and errors are costly.
Each topic carries a stance: use it, try it, watch it, or avoid it. The stance is what we would do today, based on our own projects and demos.
Start with a general NER model to learn what you need. Fine-tune a small model once your entities are specific to your domain, and handle negation and context separately.
For a high-volume, privacy-sensitive coding workflow, a pipeline of small, task-specific models beat repeated LLM calls on cost, speed, privacy and recall. We would not assume that holds for every task.
Working on something in this area? Tell us the problem and we will tell you what we would do.
View AllSend us the paragraph you'd normally write to a colleague. We'll get back to you within 24 hours.
hello@tathastha.com