(Lesson 06.06 · 18 minutes · No sensor · For everyone)

Responsible AI.

A model that works isn't enough. It has to work for everyone who'll use it, with their permission, and say clearly where it goes wrong. Here you'll see who's responsible for the mistakes, how to find the people who get left behind, and what the European rules ask for.

What you learn

Who's responsible when a model gets it wrong
How data leaves some people behind, and how to notice
When to ask for consent and when not to use a model
What the AI Act and the GDPR ask of someone who makes a small app

You need to know

If you don't remember one of these, the link takes you to the chapter that explains it.

How do you want to follow it

Preparing the scene…

01 Who's responsible
When the model gets it wrong, a person is responsible.

An app looks at a photo of a mushroom and says it's safe to eat. It's poisonous. The model^01.01 isn't to blame, because it's a file of numbers that doesn't know what it's doing. People are responsible, whoever made the app and whoever used it to decide. In Europe, the new rules on damages treat software as a product, just like a blender.

In practice. Before you publish, ask yourself what happens to the people using the app when the model gets it wrong. If the answer scares you, the model shouldn't decide on its own.
02 Skewed data
The model also learns the imbalances in the data.

You train the gesture model with your own hands. Then a friend with darker skin tries it, and it makes more mistakes. It only really knows the people it has seen. When it makes more mistakes with one group of people, it's called bias. In 2018, three systems that guess gender from a face were wrong up to 35 times out of 100 with dark-skinned women. With light-skinned men, almost never.

In practice. Before you collect the data, write down who's going to use the app. Then check that the dataset^01.05 has the hands, voices and faces of all those people, as well as your own.
03 Group by group
The average hides the people left behind.

A model that understands voice commands gets it right 95 times out of 100. With voices that have a different accent, it drops to 80. The average was true, but it mostly described the most common voices. Measuring the model group by group is called evaluation by group. In 2020, five systems that transcribe speech got an average of 35 words out of 100 wrong with African American speakers, and 19 with white speakers.

In practice. In the Evaluate step of the lab, have someone else try the model. If the accuracy^06.01 drops a lot, the dataset is missing examples like theirs.
04 Consent
Anyone who ends up in your data has to say yes.

For the voice model you record your family, and for the face model you photograph your friends. A face or a voice tells you who a person is. Before you use them, ask, and explain what they're for and where they'll end up. This free and informed yes is called consent. It can be taken back at any time, and then that person's examples leave the dataset.

In practice. Keep a list of who said yes and which examples are theirs. If someone changes their mind, you'll find their examples and delete them in a minute.
05 When not to use it
Sometimes the right choice is not to use the model.

Every model makes a few mistakes. If a mistake costs little, like getting a song wrong, that's fine. If it costs a lot, like a poisonous mushroom, the model can only advise, and a person decides. This is called human oversight. And if the data is missing the people who will use the app, or a simple rule is enough, you don't need the model at all.

In practice. For every answer the model gives, ask yourself who will notice if it's wrong, and when. If nobody notices in time, add a human check or give up on the model.
06 The limits
People using the app need to know where it goes wrong.

In the app, write that a model is answering, what it can do and where it goes wrong, with real numbers. It recognizes 9 leaves out of 10 in daylight, and far fewer in the dark. It doesn't know the diseases it has never seen. Saying these things plainly is called transparency. People who know the limits trust the app just enough, and don't use it where it doesn't work.

In practice. Put the limits next to the answer, not on a page nobody opens. One line under the result is enough, for example, it's often wrong in the dark.
07 The European rules
In Europe, the AI Act sorts uses by risk.

The European regulation on artificial intelligence is called the AI Act. Some uses are banned, like giving people a score for how they behave and then treating them worse. Others are high risk, like choosing who to hire, and they come with strict rules. A chatbot has to say that it's an AI. An app that recognizes mugs or gestures is minimal risk, and the AI Act asks nothing special of it.

In practice. Does your app recognize people or their emotions, or help make decisions about work, school or health? Read the rules before you start, because that's where the obligations are real.

01 / 07

(Go deeper)

A few more numbers.

(01)

Three measures of bias

In 2018 Joy Buolamwini and Timnit Gebru tested three systems that guess gender from a face. They used 1,270 photos balanced by gender and skin tone. The error reached 34.7 percent with dark-skinned women, and 0.8 with light-skinned men.

In 2020 Allison Koenecke and colleagues had five widely used systems transcribe 19.8 hours of interviews. On average, 35 percent of the words were wrong with African American speakers and 19 with white speakers. Even on the very same sentences, the first group got about twice as many errors.

The model card for MediaPipe's hand model measures the error on 420 photos, 35 for each pair of gender and skin tone. On average, the hand points are off by 4.9 to 7.3 percent of the size of the palm. The highest value is on the darkest tone, but there's no pattern linked to skin.

34.7%highest error, dark-skinned women
0.8%highest error, light-skinned men
35 and 19wrong words out of 100, in the two groups of speakers

(02)

The AI Act and the GDPR for a small app

The AI Act has been in force since August 1, 2024, and it arrives in stages. The bans apply from February 2, 2025. The transparency obligations, like telling people that a chatbot is answering, apply from August 2, 2026. For high-risk uses, the Digital Omnibus on AI, a regulation from July 2026, moved the dates. For uses like choosing who to hire, to December 2, 2027, and for AI inside products like medical devices, to August 2, 2028.

The GDPR protects personal data, meaning any information about a person who can be recognized. A face, a voice or a location all count. If you use them to recognize who someone is, they become biometric data, and you usually need explicit consent. Photos kept within the family stay outside the GDPR, but a published app or a shared dataset doesn't. If the data never leaves the phone and you never receive it, you have much less to protect.

(03)

Who pays for the damage

In 2024 the European Union updated its rules on defective products, with Directive 2024/2853. It applies to products placed on the market after December 9, 2026, and software counts as a product too, AI systems included. Whoever makes it is liable for harm to people, property and private data if the product was defective, even when they weren't at fault.

Software made and distributed outside a commercial activity is left out, like a lot of free and open source software. A law written specifically for AI, the AI Liability Directive, was withdrawn in 2025. For now, a model's mistakes are judged by these rules and by each country's rules on civil liability.

(Try it on your phone)

Your dataset, seen from the outside

Open the lab and scroll through one of your projects, example by example. Then check off these points. It takes ten minutes. If you don't have a project yet, think about the dataset you'd like to collect.

Open the lab

(01) The examples include at least two people besides you
(02) There are hands, voices or faces different from yours, in age, skin or accent
(03) There are different lights, rooms and times of day
(04) You tried the model with someone who isn't in the dataset
(05) Everyone who appears in the photos or recordings knows it and said yes
(06) You can say in one sentence where the model goes wrong

Every Not yet is something to do before you let other people use the model. The most useful one is usually to have someone new try the model and look at the result in the Evaluate step.

(Quiz)

4 questions to close the lesson.

When you have answered them all, the lesson counts as done and its stone on the map turns yellow. Answering is what counts, not the score.

(01)

A mushroom app says a poisonous mushroom is safe to eat. Who's responsible?
(02)

Your gesture model gets 95 percent right, but you've only tried it with your own hands. What do you do before letting other people use it?
(03)

Your sister gave you her voice for the dataset. A month later she asks you to take it out. What do you do?
(04)

Under the AI Act, what's the risk level of an app that recognizes mugs and house keys?

New words, and where they are explained

Biaschapter 2
Evaluation by groupchapter 3
Consentchapter 4
Human oversightchapter 5
Transparencychapter 6
AI Actchapter 7
Personal dataGo deeper

Sources and reading

To close the lesson, answer the quiz

← An app that works without a network The course review Back to the map

(Next lesson · 07.01) A real case, the olive leaf