(Lesson 05.02 · 22 minutes · Keyboard · First model)

Classifying and searching text on the phone.

Turn on the light, what's the weather tomorrow, put on some music. Here you'll teach the phone to understand what a sentence is asking for, with a ready-made model and a few examples of your own, and you'll use the same trick to search your notes by meaning.

What you learn

How to build a sentence classifier with just a few examples
When finding the closest example sentence is enough
How search by meaning works, and where it goes wrong

You need to know

If you don't remember one of these, the link takes you to the chapter that explains it.

How do you want to follow it

Preparing the scene…

01 The question
Understanding what a sentence asks for, among a few answers.

You want an app that understands written commands. Turn on the light, what's the weather tomorrow, put on some music. The possible sentences are endless, but the intents are few, light, weather, music and other. Recognizing the intent is a kind of text classification^01.04. Each sentence goes into one of your classes.

In practice. Choose intents that are clearly separate. If you mix up two classes yourself when you read the sentences, the model will mix them up even more.
02 The body
A ready-made model turns each sentence into numbers.

You don't need to teach the model Italian from scratch. A model already trained on millions of sentences turns each sentence into an embedding^05.01 of 384 numbers. Sentences with the same meaning end up close together, even when they use different words. It's the body of your model, and it doesn't get trained again, just like in transfer learning^01.05.

In practice. If your sentences won't be in English, choose a multilingual model. One trained only on English gives much worse embeddings in other languages.
03 The head
A small head gets trained on top.

On top of the body you put a head with one output for each intent. It receives the 384 numbers and picks the class. With four classes it has about 1,500 weights, and it trains in a moment. With ten or twenty sentences per intent, written in different ways, the model already recognizes sentences it has never seen.

In practice. Have several people write the example sentences. Everyone asks for things in their own way, and the model needs to know all those ways.
04 Without training
Sometimes finding the closest sentence is enough.

There's an even simpler way. You save the embeddings of the example sentences along with their intent. For a new sentence, you look for the most similar example and take its intent. That's nearest neighbors^01.04, applied to embeddings. Adding an intent just means writing a few more examples.

In practice. With this method you can also show why the model answered the way it did. Just show the example sentence that's most similar.
05 Search
Searching by meaning, not by matching words.

The same trick works for searching. You turn all your notes into embeddings, just once. When you search, you turn the query into an embedding too, and take the closest notes. Type "work meeting" and you'll find "appointment with the boss at three", which doesn't share a single word with it. That's semantic search.

In practice. Searching by meaning gets names, codes and exact numbers wrong. Keep it alongside a normal word search, and show the results of both.
06 Other
A class called other, for the sentences that don't fit.

Someone will type sentences that have nothing to do with it, like "I'm hungry" or "tell me a joke". Without an "other" class, the model will still send them to light, weather or music. So you add examples of unrelated sentences, and a similarity^05.01 threshold. If even the closest example is far away, the answer is other.

In practice. Also put into the "other" class sentences that sound like commands but aren't, like "the light at sunset was beautiful".
07 In the phone
It all runs in your pocket, even with no network.

A multilingual sentence model weighs about 120 megabytes in its light version, and on the phone it turns a sentence into numbers in a few dozen milliseconds. The head weighs just a few kilobytes. The sentences never leave the phone, and the command works even in airplane mode. For an app that reads your messages, that's the difference that matters most.

In practice. If 120 megabytes is too much, look for a model for a single language, or one with a smaller vocabulary. Much of the weight sits in the vocabulary itself.

01 / 07

(Go deeper)

A few more numbers.

(01)

Why the vocabulary weighs so much

A multilingual model like paraphrase-multilingual-MiniLM has a vocabulary of about 250,000 tokens, to cover dozens of languages. Each token has an embedding of 384 numbers. That makes 96 million weights just for the vocabulary, out of about 118 million in total.

So the part that understands sentences, twelve small layers, weighs less than a fifth of the total. That's why a model for a single language, with a vocabulary of 30,000 tokens, can weigh three or four times less and stay almost as good in that language.

250,000tokens in the vocabulary, roughly
96 Mweights just for the vocabulary
118 Mweights in total

(02)

The classifier in code

If you write code, all of this fits in a few lines. The examples get turned into embeddings once and saved. For each new sentence you compute its embedding, compare it with all the examples, and take the intent of the most similar one, if it's similar enough.

With a few hundred examples the comparison takes less than a millisecond. With tens of thousands of notes you use special structures called vector indexes, which find the neighbors without comparing them all.

const e = await body(sentence);                     // 384 numbers
let best = { sim: -1, intent: "other" };
for (const ex of examples) {
  const sim = cosine(e, ex.embedding);
  if (sim > best.sim) best = { sim, intent: ex.intent };
}
const answer = best.sim > 0.6 ? best.intent : "other";

(03)

Typical mistakes

Sentence embeddings mostly capture the topic. "Turn on the light" and "don't turn on the light" talk about the same thing, and they end up very close. A classifier built only on embeddings may turn on the light for someone who asked for the opposite.

If negations matter in your problem, put them in the examples as separate classes, or add a simple check for words like "not" and "never". The same goes for irony, which even big models often miss.

(Try it on your phone)

Your commands

In the lab you build a classifier for written commands, with four intents you choose. It takes ten minutes and, if you can, a second person to write test sentences.

Open the lab with the button below. It starts with the Text input already chosen, and the first time it downloads the sentence model.
Create four classes, for example light, weather, music and other.
Write 10 sentences per class, all different. In the other class, put sentences that have nothing to do with the rest.
Press Train. Then in Use, type new sentences and look at the class and the closest example sentence.
Try a negation, like "don't turn on the light", and see what happens.

Open the lab →

(Quiz)

4 questions to close the lesson.

When you have answered them all, the lesson counts as done and its stone on the map turns yellow. Answering is what counts, not the score.

(01)

Why are ten or twenty sentences per intent enough?
(02)

You type "work meeting" and find "appointment with the boss at three". Why?
(03)

You type "don't turn on the light" and the model answers light. Why does that happen?
(04)

Where is most of the weight in a multilingual sentence model?

New words, and where they are explained

Semantic searchchapter 5

Sources and reading

To close the lesson, answer the quiz

← Words become numbers Back to the map

(Next lesson · 05.03) A language model in your pocket