YouTube is getting new ‘dubbing tool’ powered by AI

YouTube announced at VidCon on Thursday that it is currently testing an AI-based tool that will assist creators in automatically dubbing their videos into other languages. Aloud, an AI-powered dubbing service that is a part of Google’s in-house incubator Area 120, collaborated with YouTube.

Support for multi-language audio tracks was added to YouTube earlier this year, allowing creators to add dubbing to both new and existing videos and expand their international audience. According to the company’s statement to TechCrunch, creators have dubbed more than 10,000 videos in over 70 languages as of June 2023.

In the past, creators had to work directly with third-party dubbing services to make their audio tracks, which can take a long time and cost a lot of money. They can dub videos for free with Aloud.

Aloud was first introduced by Google in 2022. A video is transcribed for the creator by the AI-powered dubbing product, which then translates and produces a dubbed version. Before Aloud generates the dub, creators can review and edit the transcription.

Amjad Hanif, YouTube’s vice president of creator products, told the crowd yesterday that hundreds of creators are participating in the tool’s testing. The company will soon make the tool available to all creators. At the moment, Aloud is available in Portuguese, Spanish, and English. However, additional languages, including Hindi and Bahasa Indonesian, will be offered in the future.

Hanif went on to say that YouTube is working on “working to make translated audio tracks sound like the creator’s voice, with more expression and lip sync.” TechCrunch was informed by YouTube that Aloud would soon be able to offer features like lip reanimation, improved emotion transfer, and voice preservation thanks to generative AI.