China's ByteDance has created a new way for people to immediately change their voice into another person's using generative-AI technology.
The tool, called StreamVoice, is not yet publicly available. Still, it shows the fast development of AI that enables easy and convincing audio and visual impersonations of public figures, often referred to as "deepfakes." Already this year, people have used AI to impersonate the pop star Taylor Swift and President Joe Biden as the 2024 election nears.
Several technical researchers at ByteDance, which owns the wildly popular social platform TikTok, and other researchers from Northwestern Polytechnical University, a school in China known for its work with the nation's military, created StreamVoice. Northwestern Polytechnical University is not affiliated with Northwestern University in the US.
The researchers said in a new paper that StreamVoice can do "real-time conversion" of a user's voice to any other, so long as the user has just "one utterance" of speech from the voice they want to imitate. The output occurs at the speed of livestreaming, with just 124 milliseconds of latency, the paper, which noted that AI voice-conversion technology thus far has mostly only been effective "offline," said. The researchers noted that "recent language-model advancements" allowed them to create this tool.
"Experiments demonstrate StreamVoice's ability to convert speech in a streaming manner with high speaker similarity for both seen and unseen speakers while maintaining performance comparable to non-streaming voice conversion systems," the researchers wrote.
The paper, citing Meta's Llama large language model that has grown in popularity since its initial release a year ago, says researchers built StreamVoice using "the LLaMA architecture." It also used open-source code from Meta's AudioDec, which Meta describes as a "plug-and-play benchmark for audio codec applications." Researchers trained the tool mostly on sets of Mandarin speech and one multilingual set that included English, Finnish, and German, the paper says.
The researchers did not describe how they think people should use StreamVoice, though they admitted "it may carry potential risks of misuse for various purposes, such as spreading fake information or phone fraud." They suggested people report illegal usage of voice conversion to the proper authorities.
AI experts have warned for years about the proliferation of deepfakes as the technology improves. Recently, a robocall with a deepfake of Biden told people not to vote in the New Hampshire primary. The New Hampshire Department of Justice is investigating this robocall.
Are you a tech employee or someone with a tip or insight to share? Contact Kali Hays at khays@insider.com or on the secure-messaging app Signal at 949-280-0267. Reach out using a non-work device.
