Small fine-tuned models vs per-chart LLM calls
Risk adjustment coding is a reading job. A coder opens a patient’s chart, often hundreds of pages of progress notes, scanned faxes, lab reports and medication lists, and finds every chronic condition the provider actually addressed at a visit. Each one becomes an ICD-10 code, and the codes that map to HCCs drive how a health plan is paid.
The obvious way to automate that today is to hand each page to a large language model. That is what our client’s first pipeline did. It worked. It was also slow, expensive, and sent protected health information (PHI) to a third-party API. We replaced it with a pipeline of small, fine-tuned models. This is what changed, and where we would be careful.
What the LLM pipeline cost
The original pipeline made about a hundred LLM calls per chart, and it showed in the numbers:
- About 20 minutes per chart
- About $0.31 of model spend per chart, plus GPUs for document conversion
- Every page sent to a third-party API, so PHI left the building
- 51% recall on visit dates, so codes were often not tied to the right visit
What we built instead
Instead of one large model doing everything, we split coding into the steps a human coder takes and used the smallest component that does each step well:
- Read the chart. Use the text layer first and OCR only on scans; turn tables into sentences.
- Find conditions. A compact biomedical entity-recognition model marks every disease mention.
- Find the code. Medical embeddings search the full ICD-10-CM catalogue.
- Apply rules. Deterministic, explainable guidelines handle negation, history, combination codes and excluded pairs.
- Decide like a coder. A fine-tuned model asks one question: would a coder code this?
- Find the visits. A learned model scores every date on a page to find the one that starts each visit.
The whole system ships as one container with every model inside it, running on ordinary CPUs inside the client’s own environment.
What we measured
We measured at chart level against professional coders’ results, on charts held out from all training.
| Cloud LLM pipeline | Our approach | |
|---|---|---|
| HCC (V28) recall / precision | 73% / 70% | 77% / 74% |
| Visit-date recall / precision | 51% / 74% | 83% / 75% |
| Exact ICD-10 code, recall / precision | 52% / 53% | 66% / 47% |
| Time per chart | about 20 min | about 1 min, on CPU |
| Model spend per chart | about $0.31 | $0 |
| Data sent to a third party | yes | no |
We also asked a small general-purpose medical model (MedGemma 4B) to do the job directly. It reached 31% recall and 62% precision on HCCs, and 13% of the ICD-10 codes it returned were not valid codes. Ours are 0% invalid, because every code is looked up in the catalogue rather than generated.
Where we would be careful
- Exact-code precision went down. Our pipeline outputs some extra codes, so precision at the exact-code level (47%) is lower than the LLM pipeline’s (53%). The decision model has a threshold that trades recall for precision, so a client who prefers fewer suggestions can have them.
- You need a way to measure. None of this is knowable without labelled charts coded by professionals and held out from training. If you cannot measure against that, you cannot tell a better pipeline from a worse one.
- This is one workload. The task is narrow, the volume is high and the data is sensitive. For low volume, or a task that changes every week, the simplicity of an LLM call may well win.
- Not every idea helped. Several things that sounded good, such as bigger rerankers and zero-shot verifiers, made results worse and were dropped. We keep a change only if it helps on the held-out charts.
- A person stays in the loop. The engine proposes codes with their evidence (page, sentence and the HCCs they map to) so coders review in seconds instead of re-reading the chart. It supports the coder; it does not replace them.
The pattern behind it
Most of coding is search and rules, and the judgement calls are narrow. Decompose the job, fine-tune small models for the narrow decisions, and measure every step. The disease-mention step is the same kind of biomedical entity recognition you can run in your browser on our disease detection demo, and the full pipeline is described in the case study.