It’s easy for those of us who have good hearing to take advantage of all the latest technology on our phones and computers. But what do people who have hearing loss do when they need to use apps or other programs that require sound? Tap or click here for a quick test to see how well your hearing is.
To help combat this problem, Google developed Live Captioning that works with almost all apps. Like YouTube’s closed captioning, Live Caption keeps spoken words in text format, making it easier for the hearing impaired community to enjoy content.
But the biggest difference between Google’s Captioning and YouTube’s is that Live Caption works as the name suggests – Live. It effectively transcribes any spoken words into on-screen text and works with everything from video calls to podcasts.
Here’s the backstory
The functionality is available on some Android phones for now and was first introduced with the Google Pixel 4. As Google points out, adding captions to video and spoken audio only requires one tap. Live Caption also works in the Google Chrome browser for desktop computers.
“It happens in real time and completely on-device, so it works even when you don’t have cell data or Wi-Fi, and captions are always private and never leave your phone. Whatever you do.” Searching for anything, will not find captions for it,” Google explained in a blog post.
Live Caption is also not developed specifically for the deaf community. Situations may arise where audio listening is not possible, such as on a train, bus or other public places. Can’t hear and forget your earbuds at home? Never mind, because Live Caption can help you out.
Here’s how Live Caption works
Live Caption is available on the Google Pixel 4 right now and will be available for other Pixel models later this year. It will be coming to more Android phones like Samsung’s Galaxy S series and select OnePlus smartphones. You can also use it with your Chrome browser for Windows, Mac and Linux.
How it actually works is a technological marvel. Google’s developers explained that it works through a combination of three on-device deep learning models. Of the three, there is a text-based recurrent neural network model for ambiguous punctuation. That’s what helps distinguish between subtle nuance and meaning.
“Live Caption integrates signals from the three models to form a single caption track, where sound event tags, such as [applus] and [music], appear without interrupting the flow of speech recognition results. Prediction of punctuation marks while the text is updated in parallel,” Google explained.
How to Easily Make Some Captions Live
If you have a supporting smartphone, you don’t need to download anything for it to work. It’s baked into the operating system, and you’re all set to go. You can also enable it for Google Chrome on your desktop computer.
Once you enable it, there are many other options available. You can decide where text goes on the screen, whether profanity is filtered out, or whether sound labels for music or applause are displayed. You can also adjust the size of the text as you see fit.