Most AI runs in a data centre because that is where the big models live. But for some products the cloud is the wrong place, and the reasons are usually practical, not technical.
Four questions that decide it
Does the data have to stay put? If a recording, a photo or a record must not leave the phone or the site, the model must come to the data. That settles it for many health, finance and workplace cases.
Is the network reliable where the work happens? A camera on a factory floor, a scanner in a warehouse, an app used in a lift: a feature that stops when the signal drops is a feature people stop trusting.
Does delay matter? A round trip to a server adds time. For live camera feeds or typing assistance, a model on the device answers sooner.
What does it cost at scale? Every cloud call is billed. A feature used thousands of times a day per user can cost far less when the model runs on hardware the user already owns.
What you give up
A model small enough to run on a phone is less capable than a large cloud model. It is strong at narrow tasks such as recognising an item, transcribing short speech or sorting a message. It is weaker at open-ended reasoning. Updating it also takes more care, because it lives on many devices.
The hybrid answer
Many good designs split the work. The device handles the common, simple cases instantly and privately. The cloud takes the hard ones, with a clear rule for when to hand over, and with the user’s consent where data would leave the device.
The takeaway
Decide from privacy, connectivity, delay and cost, in that order. If two or more point to the device, build for it, and keep the cloud as the backup, not the default.
We cover this in on-device and edge AI, and for strict data rules see private and sovereign deployment.
Work with us on this
- On-Device and Edge AI: Small language, speech and vision models running on phones, laptops and edge hardware, with no cloud round trip and no data leaving the device.
- Private and Sovereign AI Deployment: AI that runs where your data must stay: on your servers, in your private cloud or inside your country, with access control, audit logs and monitoring.