Understanding language, the review.
Understanding language in ten concepts and six new questions, from tokens to models that look at a photo. If an answer doesn't add up, under the explanation you'll find the chapter to read again.
The key concepts.
- Token
A piece that a language model splits text into. It can be a whole word, a syllable or a symbol. In English a word averages about 1.3 tokens, and in Italian a bit more.
- Embedding
A list of numbers that sums up the content of a photo, a sound or a sentence. Two similar things have embeddings that are close together.
- Cosine similarity
How close two lists of numbers are, measured by the angle between them. It goes from minus one to one. It's used to compare embeddings of words, sentences and photos.
- Transfer learning
Starting from a model already trained on huge amounts of data and teaching it only the last part for your problem. You need far fewer examples and far less time.
- Semantic search
Searching by meaning instead of by matching words. The texts and the question become embeddings, and you show the texts closest to the question.
- Small language model
SLM for short. A model that writes text, with anywhere from a few hundred million to a few billion parameters, small enough to fit in a phone.
- Quantization
Writing the weights with simpler numbers, for example 8-bit integers instead of 32-bit decimals. The model gets about four times smaller and faster, and it usually loses little.
- Context
The text a language model keeps in mind while it answers, meaning the conversation so far and the documents you give it. The longer it is, the more memory and time it takes.
- Hallucination
When a model confidently writes something false, for example a date, a quote or an object that isn't in the photo.
- Multimodal
A model that works with several kinds of data together, for example a photo and a written question, and answers with a sentence.
6 questions that weren't in the lessons.
They bring the course's lessons together. After each answer there's the explanation and the place to read again. The review doesn't change the map.
-
You ask a small model the same question, first in English and then in Italian. In Italian it takes a bit more memory and time. Why?
Token vocabularies were built mostly on English. The same sentence in Italian gets split into more pieces, and every extra token costs memory and time. Explained in 05.01, chapter 2 →
-
An app uses word embeddings to suggest jobs to people looking for work. What do you check before you publish it?
Embeddings also learn from texts the biases of the people who wrote them. Programmer minus man plus woman landed close to homemaker. Someone has to look for them and fix them. Explained in 05.01, Go deeper →
-
In your command classifier, someone types "where did I put my keys" and the app turns on the light. What's missing?
Without an "other" class, every sentence has to land in one of your classes. With unrelated examples and a threshold, a sentence far from every example becomes other. Explained in 05.02, chapter 6 →
-
You search your notes for an invoice number, 4471. Search by meaning shows you notes about expenses, but not that one. What should you do?
Search by meaning captures the sense, and it gets names, codes and exact numbers wrong. With a word search alongside it, you find those too. Explained in 05.02, chapter 5 →
-
Your language model weighs 3 GB. The phone moves 60 GB per second from memory. How many tokens per second can you hope for, at most?
For every token, the model rereads all its weights. 60 divided by 3 makes 20 tokens per second, and in practice you get part of that. Explained in 05.03, chapter 4 →
-
You send a photo and a one-line question to a small multimodal model. The first word takes longer than with the question alone. Why?
The little squares of the photo become embeddings that go in as tokens. A photo is worth hundreds of tokens, like a few paragraphs, and it makes you wait longer for the first word. Explained in 05.04, chapter 2 →