-
01 The question
A photo and an answer.
You point your phone at a plant and ask what species it is. To answer, it needs a model, a program that learned to recognize plants by looking at thousands of photos. The question of this lesson is simple. Where does that model run?
In practice. Every time an app gives you a smart answer, a model somewhere is doing the math. It can live in a data center, or inside the phone.
-
02 The cloud
The photo leaves, and goes a long way.
In the most common setup, the photo leaves the phone, reaches the cell tower, crosses the network and ends up in a data center. There, a powerful computer does the math and sends the answer back. It works well, as long as there's a network.
In practice. A data center can run huge models, with hundreds of billions of parameters, that would never fit in a phone. That's why the cloud exists.
-
03 Latency
The round trip takes time.
With a good signal, the trip across the network alone takes 30 to 150 milliseconds, and then there's the wait on the server. This time between question and answer is called latency. For the odd photo, that's not a problem. To follow a hand that moves thirty times a second, it is.
In practice. A video at 30 frames per second leaves 33 milliseconds for each frame. If the answer takes 150, it arrives when the hand is already somewhere else.
-
04 Privacy
Once it leaves, the photo isn't only yours anymore.
The photo is received, read and often stored on computers you don't control. For a plant, no big deal. For an X-ray, a child's voice or a customer's face, everything changes, and in Europe the GDPR comes into play.
In practice. If the computing stays on the phone, nothing leaves, so there's nothing to protect along the way.
-
05 The edge
The model moves into your pocket.
Edge means the edge of the network, where the data is born. Doing Edge AI means running the model right there, on a phone, a watch or a camera. The photo goes from the camera to the chip, and the answer arrives in a few tens of milliseconds, even in airplane mode.
In practice. You already use it. Face unlock, the keyboard that suggests your next word, live captions and searching your gallery by words all run on the phone.
-
06 The bill
Five reasons, and a price to pay.
Jeff Bier, founder of the Edge AI and Vision Alliance, sums up the advantages with an acronym, BLERP. It stands for bandwidth, latency, economics, reliability and privacy. The price is that a phone has little memory and a battery you don't want to drain, so the model has to be small.
In practice. A model built for a phone usually weighs between 1 and 50 megabytes. A large language model in a data center can reach hundreds of gigabytes.
-
07 Together
Often the answer is half and half.
Many apps do both. The phone handles the quick, frequent decisions on its own, and calls the cloud only when a bigger model is needed. In the rest of the course, you'll learn to build the part that lives in your pocket.
In practice. Ask yourself three questions before you choose. Can the answer wait a tenth of a second? Can the data leave the phone? Will the network always be there? If even one answer is no, the model goes on the device.