(Lesson 01.05 · 15 minutes · No sensor · For everyone)

The method in seven steps.

Every model in this course is built with the same seven steps, whether it recognizes a leaf, a sound or a gesture. Here you'll follow them with a real example, an olive leaf to check in the fields, where there's often no signal.

What you learn

The seven steps that come back in every hands-on lesson
Why the question comes before the technology
How to split the data so you don't fool yourself
What changes when the data is a sound, a movement or a sentence

You need to know

If you don't remember one of these, the link takes you to the chapter that explains it.

How do you want to follow it

Preparing the scene…

01 The question
First of all, what does it need to recognize?

It all starts with a simple question. Looking at a photo of an olive leaf, the phone has to say whether the plant is healthy or sick. And it has to do it out in the fields too, where the network often doesn't reach.

In practice. Before thinking about technology, write down where it will be used, on which phone and in how much light. Those answers change every choice that comes after.
02 The data
The model only learns from what you show it.

To teach the model^01.01 to recognize a sick leaf, you need lots of photos of leaves, healthy and sick. And they have to be taken where the model will really work, in the sun, in the shade, in the rain. The set of these photos is called a dataset. If they all look alike, the model learns badly.

In practice. In agriculture photos are collected over a whole year, because the leaves and the light both change with the seasons.
03 The labels
Every photo needs its right answer.

A photo on its own teaches nothing. Next to each one you have to write what it shows, healthy or sick. This answer is called a label. Sometimes you also mark where the spot is, with a box. It's like writing down the right answers to an exercise before handing it out.

In practice. The photos are split into three groups. With the first, the model studies, with the second it checks whether it's learning well, and the third stays aside for the final exam.
04 The architecture
You don't start from scratch.

The heart of the model is a neural network^01.03, a program that looks at the photo one piece at a time. The first layers notice simple things like edges and colors, the later ones recognize shapes and spots. Usually you start from a network that has already seen millions of images, and you teach it only the last part, the one that decides.

In practice. For a phone you pick lightweight networks, made to use little memory and little battery.
05 Training
You learn by making mistakes.

During training the model looks at the photos thousands of times, and on each pass it's a little less wrong. The yellow line on the chart tells you whether it's really understanding. If it stops improving while the other one keeps going down, it's learning the photos by heart. That's called overfitting.

In practice. To avoid it you show the same photos rotated, cropped or in different light, and you stop at the right moment.
06 Optimization
Making it fit in a phone.

A model that works on a computer is often too heavy for a phone. So you slim it down by using simpler numbers for its calculations, and it becomes about four times smaller. This step is called quantization. Then you test it on an ordinary phone, not the fastest one around.

In practice. That way the model works right on the phone, and the photos don't have to be sent to any server.
07 The result
One photo, one answer, even without a signal.

In the end, the model lives inside an app. Out in the field, you point at the leaf, and in an instant you know what you're looking at. The photos stay on the phone, and everything works even without a signal, just like the edge AI^01.01 from the first lesson.

In practice. It doesn't end there. The new photos coming in from the field make the next version of the model better.

01 / 07

(Go deeper)

A few more numbers.

(01)

The same steps with different data

The example uses photos, but the method stays the same when the data is something else. What changes is what you collect, how you label it and which network you pick. The table below compares four projects from the course.

Step	Leaf (photo)	Siren (sound)	Fall (motion)	Commands (text)
Question	Healthy or sick?	Is there a siren?	Did the person fall?	What is the user asking for?
Data	Photos in sun and shade	One-second clips, street and home	Two-second windows at 50 Hz	Sentences written by different people
Labels	One class per photo, sometimes a box	Siren or no siren	Fall, sitting down, walking	Light, music, weather, other
Network	MobileNet and a new head	Small network on the spectrogram	Features and a dense network	Embedding and a new head
What you measure	Accuracy and confusion matrix	False alarms per hour	Missed falls	Misunderstood commands

(02)

Why three groups of data instead of two

The validation group is for making choices. How many epochs, which learning rate, when to stop. But every time you choose by looking at those numbers, the model adapts a little to them too. You never look at the test group while you work, and it's the only one that tells you how the model will do with new data.

A rule of thumb is 70, 15 and 15 percent. With little data you split 80 and 20, and you leave the test for a collection made on another day, in different light. Watch out for nearly identical photos, like two shots in a row of the same leaf. If they end up in different groups, the model looks better than it is.

(Try it on your phone)

Your project card

Pick a problem you can recognize with one of the phone's sensors and fill in the card. It stays saved in your browser, and you'll find it again in the final project.

(Quiz)

3 questions to close the lesson.

When you have answered them all, the lesson counts as done and its stone on the map turns yellow. Answering is what counts, not the score.

(01)

According to the method, where do you start?
(02)

The validation curve stops going down while the training curve keeps going. What's happening?
(03)

What is the test group for?

New words, and where they are explained

Datasetchapter 2
Labelchapter 3
Neural network01.03 · chapter 3
Transfer learningchapter 4
Overfittingchapter 5
ValidationGo deeper
Quantizationchapter 6

Sources and reading

Progettazione, the design page the scene comes from ↗

To close the lesson, answer the quiz

← The map of models The course review Back to the map