Listening, the review.
Listening in nine concepts, from the microphone's samples to the synthetic voice, and six questions that bring the lessons together. If an answer doesn't add up, under the explanation you'll find the chapter to read again.
The key concepts.
- Sampling
Measuring sound at regular intervals and writing each measurement down as a number, the sample. For voice, models usually use 16,000 samples per second.
- Frequency
How many times per second a sound vibrates, in hertz. Fast vibrations are high sounds and slow ones are low sounds. The Fourier transform breaks a sound down into its frequencies.
- Spectrogram
A picture of sound. Time runs from left to right, frequency from bottom to top, and the color tells you how much energy there is at each point.
- Mel scale
A scale of frequencies that imitates the ear. It keeps many rows for low sounds, where we hear the differences well, and few for high ones.
- False alarms
The times a model that listens or keeps watch goes off for no reason. For models that are always on they're counted per hour, together with the real cases it missed.
- Keyword spotting
A tiny model that's always listening for a single word, like Hey Siri, and wakes up the rest of the phone only when it hears it.
- Transcription
Turning voice into written text, word by word. It's also called automatic speech recognition, or ASR. Whisper is one of the most used models.
- WER
Word error rate. The words changed, missing or added, divided by the words in the correct text. It measures how wrong a transcription is.
- Speech synthesis
Turning text into voice. The model goes from the sounds of the language to a spectrogram, and then to the wave you play.
6 questions that weren't in the lessons.
They bring the course's lessons together. After each answer there's the explanation and the place to read again. The review doesn't change the map.
-
You want to recognize a very high whistle, above 8,000 hertz. Are 16,000 samples per second enough?
You need at least two samples for each swing. With 16,000 per second you capture frequencies up to 8,000 hertz. A higher sound needs more samples, and the microphone records 48,000. Explained in 03.01, Go deeper →
-
What do the doorbell model and Whisper have in common?
First the sound becomes an image of frequencies on the mel scale. The doorbell model looks at one second at a time, Whisper at thirty seconds. What changes is what happens next. Explained in 03.04, chapter 3 →
-
You want an app that listens all day for a baby crying. What matters most when you choose the model?
A model that runs every half second has to cost almost nothing, or it drains the battery. If you need more accuracy, a second, bigger model wakes up only when the first one hears something. Explained in 03.03, chapter 2 →
-
After the keyword, your assistant sends what it hears over the network. Which mistake should you keep lower?
Every false alarm would send sentences over the network that nobody meant to send. Better a high threshold, and accept that now and then you'll have to say the word again. Explained in 03.03, chapter 5 →
-
An app for a warehouse has to understand five commands, even without a network. What do you use?
With a few fixed commands a classification is enough, like for the three sounds from home. A small model runs on the phone without a network, and it makes fewer mistakes than full transcription. Explained in 03.04, chapter 1 →
-
A transcription model has a WER of 5 percent on the news. You want to use it in a repair shop. What do you do?
The WER only holds for the sentences you measured it on. Noise and different voices change it, so you measure it with the real sentences, spoken by the people who will say them. Explained in 03.04, chapter 4 →