(Lesson 04.03 · 18 minutes · Motion sensors · First model)

Noticing what's unusual.

A fault happens rarely, and each time in a different way. Here a model learns only what normal looks like, with the phone resting on the washing machine, and flags everything that strays from it.

What you learn

Why, for faults, you learn what normal looks like instead of learning classes
Two ways to measure how strange something is
How to choose the alarm threshold
What drift is and why normal needs updating

You need to know

If you don't remember one of these, the link takes you to the chapter that explains it.

How do you want to follow it

Preparing the scene…

01 The problem
Faults are rare, and each one is different.

You want to notice when the washing machine gets unbalanced. A classifier would need lots of examples of an unbalanced washing machine, and of every other way it can break. You don't have them, and you don't want to break it just to get them. Normal washes, on the other hand, you can record as many as you like. A model for anomalies^01.04 starts right there.

In practice. When the cases you're looking for are rare or all different, think of anomalies before you think of classes. All you need are examples of when everything is fine.
02 Normal
The model only learns what normal looks like.

You rest the phone on the washing machine and record a few normal washes. Each window^04.01 of a few seconds becomes a point, described by its features^04.01, for example how strong the vibration is. The normal points gather into a cloud. The model learns the shape of that cloud, and nothing else.

In practice. Record normal in all its variations. At 30 and at 60 degrees, full and half empty, slow spin and fast spin. Normal is wider than you think.
03 The distance
Strange means far from everything you've seen.

For a new window, the model measures how far it is from the cloud of normal. The simplest way is to find the closest normal points^01.04 and measure the distance. A window close to the cloud is normal, one far away is an anomaly, whatever the cause. The model doesn't know what broke, it only knows that something is different.

In practice. A model for anomalies doesn't give the problem a name. If you need the name, keep the anomalies it finds, and later train a classifier on them.
04 The copy
A network that tries to copy what it sees.

Another method uses a network shaped like an hourglass. It takes in the window, squeezes it into a few numbers, and tries to rebuild it. Trained only on normal data, it copies normal data well. A strange vibration, though, it copies badly, because it has never seen one. The copy error becomes the strangeness score. This network is called an autoencoder.

In practice. Watch the copy error over a whole wash. It should stay low and flat while things are normal, and rise only when something happens.
05 The threshold
You decide where normal ends.

The strangeness score is a number, and you need a threshold above which the alarm goes off. You choose it by looking at the scores of normal windows the model hasn't seen. Often you take the value that only one window in a hundred goes above. That way you accept one false alarm every hundred windows, and you catch almost everything that's really off.

In practice. A single strange window could just be someone bumping into the cabinet. Raise the alarm only if several windows in a row stay strange.
06 Drift
Normal changes over time too.

After six months the washing machine vibrates a little more, the feet have come loose, and the floor isn't the same anymore. To the model, everything has become a little strange, and the alarms go up. This slow shift is called drift. Every so often the model needs updating with the new normal, without teaching it that a real fault is normal.

In practice. Save the strangeness scores together with the date. If the average creeps up over weeks, it's drift, and if it jumps in a single day, it's a fault.
07 In the factory
The same ideas keep an eye on machines.

Factories do the same thing with motors, pumps and fans. A vibration sensor is attached to the machine, and a small model right next to it decides whether the noise is the usual one. It's called predictive maintenance. The goal is to notice a bearing wearing out weeks before it breaks.

In practice. The phone is a good prototype for finding out whether a problem shows up in the vibration. If it works with the phone, a sensor that costs a few euros will do the same.

01 / 07

(Go deeper)

A few more numbers.

(01)

Why classes aren't enough

Picture a machine that breaks down once a year. With one window every two seconds, in a year you record almost sixteen million normal windows and maybe a few dozen windows of the fault. No classifier can learn from a ratio like that, and next year's fault will be different anyway.

That's why anomalies are handled the other way around. You describe one thing well, normal, which has plenty of examples. Everything that doesn't look like it becomes suspicious, even a fault nobody had ever seen.

15.8 Mtwo-second windows in a year
1fault a year

(02)

The autoencoder in practice

For vibrations a tiny autoencoder is enough. For example, 30 features go in, the network squeezes them down to 8 and then to 3 numbers, and then widens them back to 8 and again to 30. The neck of three numbers forces the network to learn only the rules of normal, instead of copying everything.

The strangeness score is the average of the squared differences between input and copy. You train it on normal windows only, for a few dozen epochs, and on the phone it weighs a few kilobytes.

strangeness = mean( (input − copy)² )
alarm if strangeness > threshold for 3 windows in a row

(03)

How to check that it works

Normal examples are enough for training, but to give the model a score you also need a few anomalies. If you don't have any, you can make some safely. A badly spread load in the drum, a fan with a coin stuck to one blade, a foot propped up on a book.

Then you count two things. How many real anomalies the model finds, and how many false alarms it gives over hours of normal running. These are the same two errors of every system that keeps watch, and the threshold shifts the balance between them.

(Try it on your phone)

The unbalanced fan

In the lab you train a model for anomalies with a desk fan or the washing machine. It takes ten minutes and a piece of tape.

Open the lab, choose the Motion input and the Anomalies task.
Rest the phone on top of the fan while it runs. With Continuous recording, record one minute at the first speed, then one minute at the second.
Press Train. The model learns only what normal looks like.
Turn off the fan and tape a coin to one blade.
Turn it back on, go to Use and press Try it live. Watch the strangeness go up, then take the coin off and watch it go down.

Open the lab →

(Quiz)

4 questions to close the lesson.

When you have answered them all, the lesson counts as done and its stone on the map turns yellow. Answering is what counts, not the score.

(01)

Why do you learn what normal looks like for faults, instead of training a classifier?
(02)

An autoencoder trained on normal data copies a new vibration badly. What does that mean?
(03)

How do you choose the alarm threshold?
(04)

After months, the alarms slowly go up, week after week. What's happening?

New words, and where they are explained

Autoencoderchapter 4
Driftchapter 6

Sources and reading

To close the lesson, answer the quiz

← Recognizing a gesture or an activity Back to the map

(Next lesson · 04.04) Predicting a number