How we cut a healthcare partner's LLM spend to $0 per chart
FINE-TUNING

Small fine-tuned models vs per-chart LLM calls

Risk adjustment coding is a reading job. A coder opens a patient’s chart, often hundreds of pages of progress notes, scanned faxes, lab reports and medication lists, and finds every chronic condition the provider actually addressed at a visit. Each one becomes an ICD-10 code, and the codes that map to HCCs drive how a health plan is paid.

The obvious way to automate that today is to hand each page to a large language model. That is what our client’s first pipeline did. It worked. It was also slow, expensive, and sent protected health information (PHI) to a third-party API. We replaced it with a pipeline of small, fine-tuned models. This is what changed, and where we would be careful.

What the LLM pipeline cost

The original pipeline made about a hundred LLM calls per chart, and it showed in the numbers:

  • About 20 minutes per chart
  • About $0.31 of model spend per chart, plus GPUs for document conversion
  • Every page sent to a third-party API, so PHI left the building
  • 51% recall on visit dates, so codes were often not tied to the right visit

What we built instead

Instead of one large model doing everything, we split coding into the steps a human coder takes and used the smallest component that does each step well:

  1. Read the chart. Use the text layer first and OCR only on scans; turn tables into sentences.
  2. Find conditions. A compact biomedical entity-recognition model marks every disease mention.
  3. Find the code. Medical embeddings search the full ICD-10-CM catalogue.
  4. Apply rules. Deterministic, explainable guidelines handle negation, history, combination codes and excluded pairs.
  5. Decide like a coder. A fine-tuned model asks one question: would a coder code this?
  6. Find the visits. A learned model scores every date on a page to find the one that starts each visit.

The whole system ships as one container with every model inside it, running on ordinary CPUs inside the client’s own environment.

What we measured

We measured at chart level against professional coders’ results, on charts held out from all training.

Cloud LLM pipelineOur approach
HCC (V28) recall / precision73% / 70%77% / 74%
Visit-date recall / precision51% / 74%83% / 75%
Exact ICD-10 code, recall / precision52% / 53%66% / 47%
Time per chartabout 20 minabout 1 min, on CPU
Model spend per chartabout $0.31$0
Data sent to a third partyyesno

We also asked a small general-purpose medical model (MedGemma 4B) to do the job directly. It reached 31% recall and 62% precision on HCCs, and 13% of the ICD-10 codes it returned were not valid codes. Ours are 0% invalid, because every code is looked up in the catalogue rather than generated.

Where we would be careful

  • Exact-code precision went down. Our pipeline outputs some extra codes, so precision at the exact-code level (47%) is lower than the LLM pipeline’s (53%). The decision model has a threshold that trades recall for precision, so a client who prefers fewer suggestions can have them.
  • You need a way to measure. None of this is knowable without labelled charts coded by professionals and held out from training. If you cannot measure against that, you cannot tell a better pipeline from a worse one.
  • This is one workload. The task is narrow, the volume is high and the data is sensitive. For low volume, or a task that changes every week, the simplicity of an LLM call may well win.
  • Not every idea helped. Several things that sounded good, such as bigger rerankers and zero-shot verifiers, made results worse and were dropped. We keep a change only if it helps on the held-out charts.
  • A person stays in the loop. The engine proposes codes with their evidence (page, sentence and the HCCs they map to) so coders review in seconds instead of re-reading the chart. It supports the coder; it does not replace them.

The pattern behind it

Most of coding is search and rules, and the judgement calls are narrow. Decompose the job, fine-tune small models for the narrow decisions, and measure every step. The disease-mention step is the same kind of biomedical entity recognition you can run in your browser on our disease detection demo, and the full pipeline is described in the case study.

Sources

KEEP READING

Related topics

Use Fine-tuning

NER: general vs fine-tuned, and when to switch

Start with a general NER model to learn what you need. Fine-tune a small model once your entities are specific to your domain, and handle negation and context separately.

Try LLMs

Small LLMs on-device: what a 0.5B model can do

A small local language model is a good fit for narrow, text-in text-out jobs where the answer is already in the input. It is a poor source of knowledge, and it needs a person checking its output.

Try RAG

RAG reranking: when a second pass pays off

Reranking is worth trying when your retrieval is close but not reliable. It is not free accuracy, and the only way to know is to measure it on your own questions.

READY TO BUILD WHAT'S NEXT?

Not sure where to start? Start with the problem.

Want this applied to your data? Tell us the problem and we will tell you what we would do.

View All
Contact us

Have a problem worth getting right?

Send us the paragraph you'd normally write to a colleague. We'll get back to you within 24 hours.

hello@tathastha.com