How we cut a healthcare partner's LLM spend to $0 per chart
TECHNOLOGY

On-device AI for real-time decision making.

We built an on-device AI stack that runs models directly on phones and edge hardware, delivering results in milliseconds, offline and without sending data to the cloud.

  • Edge AI
  • Model Optimisation
  • Mobile ML
Client
Leading technology partner
Industry
Technology
Timeline
6 months
Capabilities
AI/ML/Edge Deployment
Faster intelligence
Private by Design
THE CHALLENGE

The problem wasn't the model. It was where it had to run.

"Intelligence that waits for the cloud is intelligence that arrives too late."

The product needed instant AI-driven decisions on devices with limited power, memory and connectivity. Round trips to the cloud added delay, cost and privacy concerns.

Latency

Decisions needed in milliseconds.

Constraints

Limited memory, compute and battery.

Privacy

Sensitive data should stay on the device.

OUR APPROACH

From model to device.

We designed a pipeline that takes a cloud-trained model and delivers it efficiently to the hardware people use.

Trained Model

Cloud-trained baseline

Optimisation

Quantise, prune, distil

Device Runtime

Compile for target hardware

On-device Inference

Real-time, offline

Validation

Test accuracy and latency

Rollout

Staged, monitored updates

THE SOLUTION

Intelligence designed around the device.

We built a toolchain and runtime that compress models to fit, delivering cloud-grade intelligence on the device in the user's hands.

Real-time Inference

Results in under 50 ms.

Offline First

Works without a connection.

Model Updates

Safe over-the-air model rollouts.

Privacy by Design

Data stays on the device.

Edge AI console showing a phone running live object detection offline, latency distribution, model size reduction and a staged rollout
INSIDE THE SYSTEM

Built for speed.
Built for the edge.

Our architecture spans the training pipeline to the device runtime: efficient, secure and portable across hardware targets.

Technical details
Layered system architecture
  • Training PipelineData, training, evaluation
  • OptimisationQuantisation, pruning, distillation
  • PackagingSigned, versioned model bundles
  • Device RuntimeHardware-accelerated inference
  • TelemetryPrivacy-preserving metrics
  • Update WorkflowStaged rollout and rollback
SecureSigned models, on-device data
ScalableMillions of devices
FlexibleMultiple chipsets
ReliableRollback built in
THE RESULTS

Meaningful results at the edge.

By combining model optimisation with real device constraints, we helped our partner achieve measurable improvements.

<50ms

Inference time

Fast enough to feel instant.

100%

Offline capable

Full features without a connection.

Private

By design

Sensitive data never leaves the device.

BEFORE AFTER

From cloud round-trips to instant decisions.

The same product. The same users. A fundamentally better experience.

  • Cloud dependentEvery decision needs a connection
  • Variable latencySeconds on a weak network
  • Data leaves the devicePrivacy and compliance risk
  • Rising costsInference billed per request
  • Runs on the deviceWorks anywhere
  • Instant resultsUnder 50 ms
  • Data stays localPrivate by design
  • Predictable costNo per-request cloud bill
WHAT WE LEARNED

Smaller models can do big things.

Early collaboration between ML and device engineers mattered most. Designing for the hardware from day one avoided rework and kept accuracy where users needed it.

Understand

The device's real limits.

Collaborate

ML and device teams together.

Design for impact

Optimise for the user's moment.

READY TO BUILD WHAT'S NEXT?

Not sure where to start? Start with the problem.

Tell us what you're trying to achieve. We'll help you explore the right approach from idea to impact.

View All
Contact us

Have a problem worth getting right?

Send us the paragraph you'd normally write to a colleague. We'll get back to you within 24 hours.

hello@tathastha.com