Edgy Notebook A course by Mattia Falco
ITEN
Start
Go to Map Courses Lab Glossary How it works Troubleshooting Start →
ITEN
← Back to the map (7 courses, 30 lessons)

All courses.

The map as a list. You start with the basics, then the five intermediate courses open, and the project comes last. The lessons marked as coming soon are already planned.

01
(For everyone, start here · 5 lessons)

The Basics

What Edge AI is, what's inside a phone and how a model learns. Five lessons with no formulas that help you understand everything else.

  1. 01.01 Where AI does its thinking 12 min

    A photo takes two different trips, one to a data center and one that stays on the phone. Latency, privacy, cost and network.

  2. 01.02 What's inside your phone 15 min

    CPU, GPU and NPU, the memory that matters more than the math, the sensors and the battery. With a tool that measures yours. Measure your phone

  3. 01.03 How a model learns 18 min

    Data turned into numbers, layers, weights and loss. Why training costs a lot and using a model costs little. Teach a small network

  4. 01.04 The map of models 14 min

    Images, sound, motion, text and numbers, crossed with the tasks. Including the models that aren't neural networks. Which box is it in?

  5. 01.05 The method in seven steps 15 min

    From the question to the model inside the phone, with the example of an olive leaf. It works for every kind of model in the course.

02
(Images and video · 5 lessons)

Seeing

Classify a photo, find objects, cut them out pixel by pixel and recognize the points of hands and body. With your phone's camera.

  1. 02.01 Classifying a photo 25 min

    The body that looks and the head that decides. You train your first model on the phone in ten minutes. Lab, Photo input

  2. 02.02 A good image dataset 18 min

    Variety, balanced classes, the "nothing" class, blurry photos and duplicates. How to split the photos without cheating.

  3. 02.03 Finding objects 20 min

    Boxes, overlap and duplicates to remove. How the models of the YOLO family work. Live detector

  4. 02.04 Cutting out pixel by pixel 16 min

    Semantic and instance segmentation, and a model that estimates depth from a single photo. Live segmentation

  5. 02.05 Hands, body, face 22 min

    A ready-made model finds the 21 points of the hand. Yours learns the gestures you choose. Lab, Hand input

03
(Sound and voice · 4 lessons)

Listening

To a model, sound is an image made of frequencies. From there it can recognize noises, keywords and whole sentences.

  1. 03.01 Sound becomes an image 15 min

    Samples per second, waveform, spectrogram and mel scale. You see it live with the microphone. Live spectrogram

  2. 03.02 Recognizing a sound 22 min

    One-second windows and a small convolutional network. You teach the phone to tell three sounds apart. Lab, Sound input

  3. 03.03 The word that wakes the phone 14 min

    How a phone can always be listening without draining the battery. False alarms and missed words.

  4. 03.04 From voice to text, and back 16 min

    Transcription and speech synthesis on the device, from Whisper tiny to the live captions on your phone.

04
(Sensors and time series · 4 lessons)

Sensing motion

The accelerometer and the gyroscope measure motion a hundred times per second. With those numbers you recognize gestures and activities, spot anomalies and estimate values.

  1. 04.01 The phone senses motion 15 min

    Six axes, samples per second, windows and features. You watch the traces while you move the phone. Live traces

  2. 04.02 Recognizing a gesture or an activity 22 min

    Shaking, drawing a circle, standing still. You record the examples and train the model on the phone. Lab, Motion input

  3. 04.03 Noticing what's unusual 18 min

    A model that has only seen normal examples and flags what doesn't fit. The phone resting on the washing machine. Lab, anomalies task

  4. 04.04 Predicting a number 16 min

    Regression and forecasting. Instead of accuracy, you measure how far off you are on average. Lab, number task

05
(Text and language models · 4 lessons)

Understanding language

Words become numbers, and numbers become meaning. All the way to language models that answer even in airplane mode.

  1. 05.01 Words become numbers 16 min

    Tokens and embeddings. Why "cat" ends up close to "kitty" and far from "bolt".

  2. 05.02 Classifying and searching text on the phone 22 min

    A small model that understands what sentences mean. You train a command classifier of your own and search your notes by meaning. Lab, Text input

  3. 05.03 A language model in your pocket 20 min

    Parameters, bits, memory and tokens per second. Which model fits in your phone and what it can really do. Does it fit in my phone?

  4. 05.04 Models that see and speak 14 min

    Small models that describe a photo or answer a question about an image. Where they work and where they go wrong.

06
(Optimize and deploy · 6 lessons)

From model to app

A model that works on a computer often doesn't fit in a phone. Here you learn to measure it, shrink it and run it without a network.

  1. 06.01 Measuring for real 18 min

    Confusion matrix, precision and recall, latency, memory and battery, measured on an average phone. The threshold and the two errors

  2. 06.02 Quantizing 16 min

    From 32 to 8 and then 4 bits. How much space you save and how much accuracy you lose. How many bits do you need?

  3. 06.03 Pruning, distilling, choosing the architecture 16 min

    Removing the weights you don't need, and having a small model learn from a big one.

  4. 06.04 Formats and engines 18 min

    LiteRT, Core ML, ONNX Runtime, ExecuTorch, MediaPipe and WebGPU. Which ones use the NPU, and when.

  5. 06.05 An app that works without a network 20 min

    Installable apps, privacy, updates and models that learn on the device. Lab, final step

  6. 06.06 Responsible AI 18 min

    Who answers for the mistakes, data that leave some people behind, consent, the limits to tell people and the European rules.

07
(From question to result · 2 lessons)

Final project

A real case followed from start to finish, then yours, with a card to fill in and a checklist.

  1. 07.01 A real case, the olive leaf 20 min

    OliveSense from the idea to the phone in the countryside, with the choices made along the way and what's still missing.

  2. 07.02 Your project 30 min

    Pick a problem, fill in the card, build the model in the Lab and check that you haven't skipped anything.