(Lesson 04.01 · 15 minutes · Motion sensors · For everyone)

The phone senses motion.

Inside your phone, a mass smaller than a grain of sand feels every push. Here you'll find out how the accelerometer and the gyroscope turn motion into six traces, and you'll watch your own as you move the phone.

What you learn

What the accelerometer and the gyroscope measure, and why gravity always shows up
What a window is, and what the features of a window are
Why motion sensors are perfect for small models

You need to know

If you don't remember one of these, the link takes you to the chapter that explains it.

How do you want to follow it

Preparing the scene…

01 The accelerometer
The phone feels pushes in three directions.

Inside the phone there's a tiny mass hanging from silicon springs. When the phone speeds up, the mass stays behind for a moment and the springs bend. The chip measures how much, in three directions, x, y and z. That's the accelerometer. Even lying still on the table it reads something, gravity, about 9.8 meters per second squared, pointing down.

In practice. Put the phone down on the table and watch the traces in the exercise below. The axis that reads about 9.8, plus or minus, is the one at right angles to the table.
02 The gyroscope
The gyroscope feels rotation.

The gyroscope measures how fast the phone turns around its three axes, in degrees per second. Together, the accelerometer and the gyroscope give you six numbers, the six axes of motion. With all six you can tell a shake from a rotation, and a step from a turn of the wrist.

In practice. The gyroscope uses more power than the accelerometer. If your model works with just the three axes of the accelerometer, your battery will thank you.
03 The traces
Dozens of readings per second, for each axis.

Motion sensors don't measure all the time. They take a reading at regular intervals, from 50 to a few hundred times per second. In the browser you usually get about sixty. Lined up, they form six traces, one per axis, that rise and fall as you move the phone. Walking draws a wave that repeats with every step.

In practice. 50 readings per second are enough for hand gestures. More readings mean more numbers to go through, and rarely a better model.
04 The windows
You look at a two-second slice at a time.

A gesture doesn't fit in a single reading, it lasts a moment. So the model looks at a window, which is a slice of the traces, say two seconds long. With 50 readings per second and six axes, the window is a tensor^01.03 of 100 rows by 6 columns. Then the window slides forward half a second, and the model answers again.

In practice. The window has to hold the whole gesture. Time your slowest gesture, and pick a window a little longer than that.
05 The features
A few numbers that sum up the window.

Instead of giving the model all 600 readings, you can work out a few numbers that sum them up. The average of each axis, how much it swings, the highest and the lowest value, how many peaks there are. They're called features. With about thirty features, a very small model is enough.

In practice. Before training, put the features of your classes side by side. If a single one already separates them, like the average of one axis, you don't need a complicated model.
06 Gravity
Gravity gets separated from motion.

Gravity is always inside the readings, and it moves to another axis when you turn the phone. The same gesture made with the phone upright or sideways gives different traces. That's why you take it out, with a filter that keeps only the fast changes. The phone already does this on its own, and in the browser it gives you both versions, with and without gravity.

In practice. Record your examples holding the phone in every way the people using it will hold it. In the hand, in a pocket, in a backpack.
07 In your pocket
The sensor stays on all day.

Unlike the camera, motion sensors use very little power and can stay on all the time. The phone's step counter works this way, with a small separate processor that keeps counting even with the screen off. A watch does the same thing to notice a fall. For small models, it's the ideal place to be.

In practice. In the browser the sensors only work while the page is open and in the foreground. To work with the screen off, you need an installed app, not a web page.

01 / 07

(Go deeper)

A few more numbers.

(01)

How an accelerometer is made

The accelerometer in your phone is a MEMS, a micro electro mechanical system. On a piece of silicon smaller than a millimeter there's a moving mass with lots of tiny teeth, facing other teeth that don't move. When the mass shifts, the gap between the teeth changes, and the chip measures the difference as a change in electric charge.

It usually measures up to 8 or 16 times gravity, enough for a fall or a jump. It costs a few cents and uses a fraction of a milliwatt. That's why you find one in every phone, watch, earbud and game controller.

9.81meters per second squared, which is 1 g
16 gthe most that many accelerometers can measure

(02)

The most common features

In 2012 a team at the University of Genoa published UCI HAR, one of the most widely used datasets for activities. Thirty people wore a phone on their belt and did six activities, among them walking, climbing stairs and sitting. The phone took 50 readings per second, cut into windows of 2.56 seconds. For each window the team worked out 561 features.

In practice you need far fewer. The table shows the ones people use almost every time. The typical mistake of models on this dataset is mixing up sitting and standing, because a phone on the belt stays still in both cases.

Feature	What it tells you	Good for
Average of each axis	How the phone is turned	Telling lying, sitting and standing apart
Swing	How much it moves	Still or moving
Minimum and maximum	The hardest hits	Falls, shakes, bumps
Number of peaks	How many times the movement repeats	Steps, repetitions
Main rhythm	How fast the rhythm is	Walking or running

(03)

The sensors in the browser, in code

In the browser the sensors arrive through the devicemotion event. On iPhone you first have to ask for permission, and the request has to come from a tap. On Android they arrive right away on https pages. Each event carries the acceleration with and without gravity and the rotation rate, plus the time between one reading and the next.

The exercise in this lesson does exactly that. It listens for the event, keeps the last two seconds of readings, and works out a few features five times per second.

if (DeviceMotionEvent.requestPermission) await DeviceMotionEvent.requestPermission();
addEventListener("devicemotion", e => {
  const a = e.accelerationIncludingGravity;   // x, y, z in m/s²
  const g = e.rotationRate;                    // alpha, beta, gamma in degrees per second
  samples.push([a.x, a.y, a.z, g.alpha, g.beta, g.gamma]);
});

(Try it on your phone)

Live traces

Pick up your phone and press the button. You'll see the six traces scroll by, the accelerometer on top and the gyroscope below. Underneath you'll find a few features worked out on the last two seconds. Try shaking the phone, turning it, walking around.

The tool needs JavaScript turned on.

On iPhone the phone asks for permission to use the motion sensors, so tap Allow. A computer has no motion sensors, so this only works on the phone. The readings never leave the page.

(Quiz)

4 questions to close the lesson.

When you have answered them all, the lesson counts as done and its stone on the map turns yellow. Answering is what counts, not the score.

(01)

Your phone is lying still on the table. What does the accelerometer read?
(02)

With 50 readings per second and six axes, how many numbers are in a two-second window?
(03)

Why do you work out features instead of using all the readings?
(04)

Your model only recognizes a gesture when the phone is upright. What should you do?

New words, and where they are explained

Accelerometerchapter 1
Gyroscopechapter 2
Windowchapter 4
Featureschapter 5

Sources and reading

To close the lesson, answer the quiz

← From voice to text, and back Back to the map

(Next lesson · 04.02) Recognizing a gesture or an activity