How we cut a healthcare partner's LLM spend to $0 per chart
TOPIC AREA

Fine-tuning

When to fine-tune a small model instead of prompting a large one, how to measure the difference, and what we have shipped.

Fine-tuning means training a model further on examples of your own task, so that it does one job well. For narrow, high-volume work it can beat prompting a large model on cost, speed and privacy. It only pays off if you can prove it, which is why most of what we write here is about measurement.

These notes cover when a small fine-tuned model is the right tool, how to build the held-out test set that shows whether it worked, and what to keep outside the model, such as rules for negation and history. Our examples come mostly from clinical text, where the entities are specific, the data is sensitive and errors are costly.

Each topic carries a stance: use it, try it, watch it, or avoid it. The stance is what we would do today, based on our own projects and demos.

Use

We use this on real projects and recommend it where it fits.

Use

NER: general vs fine-tuned, and when to switch

Start with a general NER model to learn what you need. Fine-tune a small model once your entities are specific to your domain, and handle negation and context separately.

Use

Small fine-tuned models vs per-chart LLM calls

For a high-volume, privacy-sensitive coding workflow, a pipeline of small, task-specific models beat repeated LLM calls on cost, speed, privacy and recall. We would not assume that holds for every task.

READY TO BUILD WHAT'S NEXT?

Not sure where to start? Start with the problem.

Working on something in this area? Tell us the problem and we will tell you what we would do.

View All
Contact us

Have a problem worth getting right?

Send us the paragraph you'd normally write to a colleague. We'll get back to you within 24 hours.

hello@tathastha.com