Glossary.
The words of the course explained in two lines. Each one takes you to the chapter that explains it, so you review only the part you need.
- Accelerometer
The sensor that measures the pushes on the phone in three directions, x, y and z. When the phone is still it measures gravity, about 9.8 meters per second squared, pointing down.
04.01 The phone senses motion, chapter 1 →- Accuracy
The share of right answers over all the test examples. It's the simplest score, but with imbalanced classes it's misleading, and on its own it doesn't tell you which mistakes the model makes.
06.01 Measuring for real, chapter 1 →- AI Act
The European regulation on artificial intelligence. It sorts uses by risk, bans some of them, sets strict rules for the high-risk ones and asks for transparency from chatbots and generated content.
06.06 Responsible AI, chapter 7 →- Anomaly
A piece of data that doesn't look like the ones seen during training. An anomaly model learns what normal looks like and flags whatever strays from it, without needing any examples of faults.
01.04 The map of models, chapter 6 →- Artificial intelligence
The broad name for programs that do things that usually take a person, like recognizing an object, understanding a sentence or answering a question. Today almost all of it is built with machine learning.
01.04 The map of models, Go deeper →- Autoencoder
A network that squeezes the input into a few numbers and tries to rebuild it. Trained on normal data, it copies strange data badly, and the error of the copy becomes the anomaly score.
04.03 Noticing what's unusual, chapter 4 →
- Bias
When a model gets things wrong more often with one group of people, for example voices with an accent or darker skin. It usually happens because that group was rare in the data.
06.06 Responsible AI, chapter 2 →- BLERP
The acronym Jeff Bier uses for the reasons behind Edge AI. Bandwidth, latency, economics, reliability and privacy.
01.01 Where AI does its thinking, chapter 6 →
- Calibration
Running about a hundred real examples through the model before quantizing it, to find the range of values between the layers and put the steps in the right place.
06.02 Quantizing, chapter 5 →- Class
One of the possible answers of a model that classifies. Healthy and sick are two classes, and so are walking, running and standing still.
01.03 How a model learns, chapter 4 →- Classification
The task of picking one answer from a fixed list of classes by looking at the whole input. Healthy or sick, dog or cat, walking or running.
01.04 The map of models, chapter 2 →- Cloud
Computers in a data center that you reach over the network. They have lots of power and memory, but the data has to travel to them and back.
01.01 Where AI does its thinking, chapter 2 →- Confidence
The percentage the model gives to an answer. It tells you how much the model prefers that answer over the others it knows, not how likely it is to be right.
02.01 Classifying a photo, chapter 6 →- Confusion matrix
A table that crosses the right answers with the ones the model gave. The hits sit on the diagonal and the mix-ups sit off it, for example sick taken for healthy.
06.01 Measuring for real, chapter 2 →- Consent
A person's free and informed yes to using their photos, their voice or other data of theirs for a specific purpose. It can be taken back at any time.
06.06 Responsible AI, chapter 4 →- Context
The text a language model keeps in mind while it answers, meaning the conversation so far and the documents you give it. The longer it is, the more memory and time it takes.
05.03 A language model in your pocket, chapter 5 →- Convolution
A small filter, for example 3 by 3 pixels, that slides over the whole image and lights up where it finds its pattern. A network made of layers of filters is called convolutional, or CNN.
02.01 Classifying a photo, chapter 3 →- Cosine similarity
How close two lists of numbers are, measured by the angle between them. It goes from minus one to one. It's used to compare embeddings of words, sentences and photos.
05.01 Words become numbers, chapter 5 →- CPU
The phone's general-purpose processor. It has a few very fast cores that run one instruction after another. It can do anything, but it runs neural networks slowly.
01.02 What's inside your phone, chapter 2 →
- Data augmentation
Changing the examples a little at random during training, for example by rotating a photo or making it brighter. The model sees more variety without you having to collect more data.
02.02 A good image dataset, chapter 7 →- Data leakage
When nearly identical examples end up both among the ones for learning and among the ones for testing. The model recognizes them, and the score looks better than it will really be.
02.02 A good image dataset, chapter 6 →- Dataset
The set of examples you use to train and check a model. Photos, audio clips, sensor recordings or sentences, each one with its own label.
01.05 The method in seven steps, chapter 2 →- Decision tree
A model made of yes or no questions about the data, one after the other, until it reaches an answer. It's small and fast and you can read it, which is why it does so well on microcontrollers and with sensor data.
01.04 The map of models, Go deeper →- Depth estimation
A model that gives each pixel a distance instead of a class. From a single photo it estimates what's in front and what's behind, but not the exact meters.
02.04 Cutting out pixel by pixel, chapter 6 →- Distillation
A small model, the student, learns to imitate the answers of a big model, the teacher. It often does better than the same small model trained on its own.
06.03 Pruning, distilling, choosing the architecture, chapter 4 →- Drift
When the data changes slowly over time, for example a machine that gets old. The model starts making mistakes and has to be updated with the new data.
04.03 Noticing what's unusual, chapter 6 →
- Edge AI
Running a model on the device where the data is created, like a phone, a watch or a camera, instead of in a data center.
01.01 Where AI does its thinking, chapter 5 →- Embedding
A list of numbers that sums up the content of a photo, a sound or a sentence. Two similar things have embeddings that are close together.
05.01 Words become numbers, chapter 4 →- Encoder and decoder
The two halves of a network. The encoder squeezes the input and works out what's in it, and the decoder widens it again to produce the output, for example a mask as big as the photo.
02.04 Cutting out pixel by pixel, chapter 3 →- Epoch
One full pass over the training data. A model usually goes through anywhere from a few dozen to a few hundred.
01.03 How a model learns, chapter 6 →- Evaluation by group
Measuring the model separately for each group of people who will use it, instead of only on average. It shows who gets left behind and which examples are missing.
06.06 Responsible AI, chapter 3 →
- False alarms
The times a model that listens or keeps watch goes off for no reason. For models that are always on they're counted per hour, together with the real cases it missed.
03.02 Recognizing a sound, chapter 5 →- Features
A few numbers computed from a window that sum it up, like the average of each axis, how much it swings or how many peaks there are. They make very small models possible.
04.01 The phone senses motion, chapter 5 →- Federated learning
Many devices improve the same model without sending out their own data. Each one learns locally and sends only the changes to the weights.
06.05 An app that works without a network, chapter 6 →- Forecasting
Estimating the next values of a series by looking at the past ones, for example how much battery will be left tonight. It's a regression where the input is the past.
04.04 Predicting a number, chapter 6 →- Frequency
How many times per second a sound vibrates, in hertz. Fast vibrations are high sounds and slow ones are low sounds. The Fourier transform breaks a sound down into its frequencies.
03.01 Sound becomes an image, chapter 4 →
- Generative model
A model that produces something new, text, voice or images, one piece at a time. It's the heaviest kind of model that runs on a phone today.
01.04 The map of models, chapter 7 →- Gesture and activity
A gesture has a start and an end, like shaking the phone. An activity goes on over time, like walking. This changes how long the window has to be.
04.02 Recognizing a gesture or an activity, chapter 1 →- GPU
The graphics processor. It has hundreds of small calculators working together, built to draw the pixels on the screen. It does the multiplications of neural networks much faster than the CPU.
01.02 What's inside your phone, chapter 3 →- Gradient descent
The way a model learns. For each weight it works out which way the loss goes down and moves the weight a small step in that direction, millions of times.
01.03 How a model learns, chapter 6 →- Gyroscope
The sensor that measures how fast the phone turns around its three axes, in degrees per second. Together with the accelerometer it gives the six axes of motion.
04.01 The phone senses motion, chapter 2 →
- Hallucination
When a model confidently writes something false, for example a date, a quote or an object that isn't in the photo.
05.03 A language model in your pocket, chapter 6 →- Human oversight
A person checks the model's answers and makes the decision, especially when a mistake can cause harm. The model advises, the person chooses.
06.06 Responsible AI, chapter 5 →
- Imbalanced classes
When one class has many more examples than the others. A model can look accurate by always giving the biggest class, so you have to check the mistakes class by class.
02.02 A good image dataset, chapter 3 →- Inference
Using a model that's already trained to give an answer about new data. It's the work the phone does when it recognizes something.
01.03 How a model learns, chapter 7 →- Inference engine
The program that reads a model file and does the math on the device, choosing between CPU, GPU and NPU. LiteRT, Core ML, ONNX Runtime and ExecuTorch are engines.
06.04 Formats and engines, chapter 1 →- Instance segmentation
A segmentation that separates each single object with its own mask, instead of coloring all the objects of the same class the same way.
02.04 Cutting out pixel by pixel, chapter 4 →- IoU
Intersection over union. How much two boxes overlap, from 0 to 1. It's used to decide whether a box found by the model matches the right one.
02.03 Finding objects, chapter 5 →
- Keypoints
Precise points on a body or an object, like the 21 joints of the hand. A model finds them and gives back their coordinates.
01.04 The map of models, chapter 3 →- Keyword spotting
A tiny model that's always listening for a single word, like Hey Siri, and wakes up the rest of the phone only when it hears it.
03.03 The word that wakes the phone, chapter 1 →
- Label
The right answer attached to an example, which is what the model has to learn to say. It can be a class, a box, an outline or a number.
01.05 The method in seven steps, chapter 3 →- Latency
The time between the question and the answer, measured in milliseconds. To keep up with a video at 30 frames per second it has to stay under 33 ms.
01.01 Where AI does its thinking, chapter 3 →- Learning rate
How much the weights get corrected at each step of training. Too high and the model jumps all over the place, too low and it learns very slowly.
01.03 How a model learns, Go deeper →- LiteRT
Google's engine for running models on phones and small devices. Until 2024 it was called TensorFlow Lite, and its files have the .tflite extension.
06.04 Formats and engines, chapter 2 →- Loss
The number that measures how wrong the model is on the examples. It's also called the loss function. All of training is about bringing it down.
01.03 How a model learns, chapter 5 →
- Machine learning
A way of building programs where nobody writes the rules by hand. You show many examples with the right answer and the program finds the rules on its own. Neural networks are the most common kind.
01.04 The map of models, Go deeper →- MAE
Mean absolute error. The average of the differences between the estimate and the true value, ignoring the sign. It measures the error of a regression in the same unit as the answer.
04.04 Predicting a number, chapter 3 →- Mel scale
A scale of frequencies that imitates the ear. It keeps many rows for low sounds, where we hear the differences well, and few for high ones.
03.01 Sound becomes an image, chapter 6 →- Model
A program that learned from examples how to turn some data into an answer. Inside are the weights, the numbers it adjusted during training.
01.01 Where AI does its thinking, chapter 1 →- Model card
A short page that comes with a model. It says what it's for, what data it was trained on, how it was measured, where it goes wrong and where it shouldn't be used.
07.02 Your project, chapter 7 →- Models in a row
Several small models working one after the other, each with a narrow task, for example checking the photo, working out what it shows and then giving the diagnosis.
07.01 A real case, the olive leaf, chapter 4 →- Multimodal
A model that works with several kinds of data together, for example a photo and a written question, and answers with a sentence.
05.04 Models that see and speak, chapter 1 →
- Nearest neighbors
k-NN for short. A model that learns no weights. For new data it looks up the most similar examples it has in memory and repeats their answer. It works well with few examples.
01.04 The map of models, Go deeper →- Neural network
A model made of layers of simple calculations linked together. Each layer transforms the numbers it gets and passes the result on to the next.
01.03 How a model learns, chapter 3 →- NMS
Non-maximum suppression. Among the boxes that overlap too much it keeps the one with the highest confidence and deletes the others, so one box is left for each object.
02.03 Finding objects, chapter 6 →- Normalization
Bringing the data to a common scale before giving it to the model. For the hand points you put the wrist at zero and divide by the size of the hand, so only the shapes are left.
02.05 Hands, body, face, chapter 5 →- Nothing class
An extra class with examples of when the object isn't there, like the empty table or a hand passing by. It gives the model a right answer for those moments too.
02.02 A good image dataset, chapter 4 →- NPU
Neural processing unit. The part of the phone's chip made specially for the multiplications in neural networks. It uses much less power than the CPU for the same work.
01.02 What's inside your phone, chapter 4 →
- Object detection
The model finds the objects in an image and draws a box around each one, with its class.
01.04 The map of models, chapter 3 →- On-device training
Continuing to train a model on the phone, with the data of the person using it. Usually only the last layers get updated, at night and while charging.
06.05 An app that works without a network, chapter 5 →- Overfitting
When a model learns the training examples by heart instead of the general rule. It does great on the data it has seen and badly on new data.
01.05 The method in seven steps, chapter 5 →
- Parameter
One of the numbers a model adjusts while it learns, also called a weight. MobileNet has about 3.5 million, a small language model from one to four billion.
01.03 How a model learns, chapter 2 →- Personal data
Any information about a person who can be recognized, like a face, a voice or a location. In Europe the GDPR protects it.
06.06 Responsible AI, Go deeper →- Precision and recall
Precision tells you how often the model is right when it says yes. Recall tells you how many of the real cases it manages to find. Usually when one goes up the other goes down.
06.01 Measuring for real, chapter 3 →- Pruning
You remove the weights and channels that matter little. The result is a smaller and faster model.
06.03 Pruning, distilling, choosing the architecture, chapter 2 →- PWA
Progressive web app. A web page that you add to your home screen, that opens full screen and works without a network too, without going through the app stores.
06.05 An app that works without a network, chapter 1 →
- Quantization
Writing the weights with simpler numbers, for example 8-bit integers instead of 32-bit decimals. The model gets about four times smaller and faster, and it usually loses little.
01.05 The method in seven steps, chapter 6 →
- RAM
The phone's working memory, from 6 to 16 GB. It holds the model's numbers while it runs. How fast they can be read from there often matters more than the power of the chip.
01.02 What's inside your phone, chapter 5 →- Regression
A task where the answer is a number instead of a class. How many steps, how many degrees, how full.
01.04 The map of models, chapter 5 →
- Sampling
Measuring sound at regular intervals and writing each measurement down as a number, the sample. For voice, models usually use 16,000 samples per second.
03.01 Sound becomes an image, chapter 2 →- Segmentation
The model decides which class each single pixel belongs to. The result is a mask that cuts the objects out along their outline.
01.04 The map of models, chapter 4 →- Semantic search
Searching by meaning instead of by matching words. The texts and the question become embeddings, and you show the texts closest to the question.
05.02 Classifying and searching text on the phone, chapter 5 →- Sensor
A component that measures something about the world and turns it into numbers. A phone has more than ten, from the camera to the accelerometer.
01.02 What's inside your phone, chapter 6 →- Service worker
A small program the browser installs next to a page. It saves pages, files and models and hands them back even without a network.
06.05 An app that works without a network, chapter 2 →- Shortcut
When a model learns a detail that comes along with the answer instead of the answer itself, for example the snow instead of the wolf. On new data without that detail it gets things wrong.
02.02 A good image dataset, chapter 1 →- Small language model
SLM for short. A model that writes text, with anywhere from a few hundred million to a few billion parameters, small enough to fit in a phone.
05.03 A language model in your pocket, chapter 1 →- SoC
System on a chip. The phone's main chip, which holds the CPU, GPU, NPU and other parts in a piece of silicon about the size of a fingernail.
01.02 What's inside your phone, chapter 1 →- Spectrogram
A picture of sound. Time runs from left to right, frequency from bottom to top, and the color tells you how much energy there is at each point.
03.01 Sound becomes an image, chapter 5 →- Speech synthesis
Turning text into voice. The model goes from the sounds of the language to a spectrogram, and then to the wave you play.
03.04 From voice to text, and back, chapter 6 →
- Tensor
A table of numbers with one or more dimensions. A 224 by 224 pixel color photo is a tensor with 224 rows, 224 columns and 3 colors.
01.03 How a model learns, chapter 1 →- TinyML
Models that run on microcontrollers, chips with a few hundred kilobytes of memory that use milliwatts. It's the smallest end of Edge AI.
01.01 Where AI does its thinking, Go deeper →- Token
A piece that a language model splits text into. It can be a whole word, a syllable or a symbol. In English a word averages about 1.3 tokens, and in Italian a bit more.
05.01 Words become numbers, chapter 2 →- TOPS
Trillions of operations per second. The number manufacturers use to describe the power of an NPU. On its own it says little, because how fast the memory delivers data matters too.
01.02 What's inside your phone, chapter 4 →- Training on the steps
Quantization aware training. During training you simulate numbers with few bits, so the model learns to work well even after quantization.
06.02 Quantizing, chapter 6 →- Transcription
Turning voice into written text, word by word. It's also called automatic speech recognition, or ASR. Whisper is one of the most used models.
03.04 From voice to text, and back, chapter 1 →- Transfer learning
Starting from a model already trained on huge amounts of data and teaching it only the last part for your problem. You need far fewer examples and far less time.
01.05 The method in seven steps, chapter 4 →- Transparency
Telling the people who use the app that a model is answering, what it can do and where it goes wrong, in plain words and with real numbers.
06.06 Responsible AI, chapter 6 →
- Validation
The group of data the model doesn't learn from, used during training to check whether it's generalizing.
01.05 The method in seven steps, Go deeper →
- WebGPU
The modern way a web page uses the device's graphics card. It lets neural networks run in the browser much faster than on the CPU.
06.04 Formats and engines, chapter 6 →- WER
Word error rate. The words changed, missing or added, divided by the words in the correct text. It measures how wrong a transcription is.
03.04 From voice to text, and back, chapter 4 →- Window
A piece of signal, for example two seconds long, that the model looks at all at once. The window slides forward in regular steps and the model answers at each step.
04.01 The phone senses motion, chapter 4 →