Artificial intelligence, fortunately, does not get talked about only in negative or polemical terms.
Why negative or controversial? Because, in addition to the detractors regardless, there are also those who have shown little inclination to accept that AI is regulated. We are referring, for example, to Sam Altman, CEO of OpenAI, who according to a very recent Time scoop would have even succeeded in having the EU AI Act amended so that his products would not be “high-risk”.
But, we said, artificial intelligence is also showing ever new and virtuous uses. In an article that we could almost define as pioneering, we wrote about how deepfakes could change dubbing in cinema. How? Making the lips of the actors more credible once a film is translated into other languages.
Since then, much more has happened: Bruce Willis has appeared in commercials without ever having gone to the set in person. And the generative intelligence that produces new images, thanks to software like Midjourney, has shown us some beautiful ones: from Pope Francis in a duvet to Donald Trump behind bars. Up to an impressive preview of the funeral of Silvio Berlusconi.
Now it is YouTube that will exploit AI for dubbing in a new and fascinating way. Let’s see how.
YouTube and AI dubbing
Try Amazon Music for your music, 30 days Free, cancel anytime
The announcement was made on Thursday, June 22 during VidCon, the annual convention for creators held at the Anaheim Convention Center in Anaheim, California.
YouTube, to improve the dubbing of its videos, will rely on generative AI.
Let’s find out how artificial intelligence will enter the platform owned by Google.
Aloud: what it is, how it works
The magic word is Aloud. Yes, because everything starts from this product which is part of Google’s Area 120 incubator, created by the Mountain View company to create highly innovative experimental products.
How will Aloud work? How will YouTube AI help with video dubbing in other languages?
There will be four steps: first the system transcribes the recorded video from a source language to a target one. On the transcription, the user (or more precisely the creator) can intervene to modify any translation inaccuracies. At this point Aloud dubs the video into the target language, based on the transcript. Finally, the video is ready to be published.
The languages available
The service is currently in an experimental phase, and is involving a hundred creators.
For now there is only the possibility to dub the video using English as source language, and Spanish and Portuguese as target languages. But Google says that “other languages will be available soon”.
An implementation is planned for 2024. It will also come to what we were talking about at the beginning, lip sync, and more. You can also pick up the person’s voice in the video and increase the expressiveness of the audio track.
Already today, in a small number of pilot videos, dubbing via AI (but not in Italian) can be experienced on YouTube.
A new opportunity for creators
Dubbing YouTube videos using Generative AI will be a major new tool in the hands of creators.
A study published in January, and conducted by YouTube, tells us that videos presented in multiple languages increase viewing time, which 15% comes from the non-source language.
Then there is the question of practicality: the audience for a particular video would be greatly multiplied, without however having to increase the channels of origin of the content: the original video alone would be sufficient.
Furthermore, creators would have – in the near future – another string to their bow: the vice president of products for YouTube creators Amjad Hanif, declared that in 2024 the platform will probably allow “translated audio tracks to sound like the creator’s voice, with more expressiveness and lip-syncing.”
A boomerang for language learning?
As is always the case with news previews, opposing interpretations of YouTube’s use of AI to dub its videos are already spreading.
On the one hand, many agree on the ready usefulness of the tool. On the other hand, there are those who complain, not without some reason, about a possible problem: videos translated into your own language would prevent you from training your ear for foreign languagesespecially English, a now universal language used in the vast majority of content on social media.
And in the sense of the lack of propensity for foreign languages, unfortunately, our country is teaching.