The media recently reported that ElevenLabs, an American artificial intelligence AI voice cloning startup, raised $80 million in a Series B funding round, making the company's valuation now at $1.1 billion - becoming a unicorn start-up worth more than $1 billion. ElevenLabs' technology is now used by 41% of employees in Fortune 500 companies.
ElevenLabs specializes in products that allow users to convert text to speech and instantly generate natural AI speech in any language. In addition, the company has developed an AI tool that allows users to create their own AI voice clones with just a few minutes of audio. In addition to the funding announcement, ElevenLabs also announced several upcoming new products, including a voice-over studio, a voice library marketplace, and a mobile app reader in preview that converts text and URLs to audio in real-time, with a new model for improved speed and language coverage.
One of ElevenLabs' core priorities is the safe and responsible development of AI technology, and they will focus on using detection technology to ensure that all AI-generated content is clearly identified. To that end, they launched an AI speech classifier last year to verify that audio samples contain content generated by ElevenLabs.
Here, you can try to understand ElevenLabs' technical strength in the field of AI speech synthesis with the following product application example.
ElevenLabs has now launched a fully automated AI voice-over or video translation tool (voice-over studio) AI Dubbing & Video Translator. All you need to do is upload a video or paste a link to it, and the tool will automatically translate the video into 29 languages in tens of seconds to minutes. What's even more powerful is that you can directly clone the voice in the original video to dub the video. Even if there are multiple people talking in the video, all of them can be cloned and translated.
There are internet users who have already experienced ElevenLabs' products. The Internet user said that ElevenLabs is by far the most affordable multilingual speech synthesis AI he has ever used! It requires a $22 monthly Creator subscription to use Dubbing, which includes 2 hours (100,000 words), and charges $0.3 per 1,000 words after 2 hours. However, judging from the current cost of computing power, the availability of AI speech synthesis is much greater than that of AI video synthesis, which is one of the cores of human communication.
The following video is an excerpt from Nvidia CEO Jensen Huang's speech on "ChatGPT's iPhone Moment" last February, ElevenLabs completed the Chinese dubbing in a dozen seconds, and cloned Huang's original voice and tone of speech.
Loading...
The following video is the official Chinese video translated by ElevenLabs:
Loading...
Original English version of this video:
Loading...
AI Dubbing & Video Translator的主要功能如下:
1. Video translation and dubbing: Translate the sound of the video from one language to another, and keep the voice characteristics of the original speaker unchanged.
2. Wide application: It can be used in a variety of occasions, such as allowing audiences of different languages to understand foreign language videos, or making the company's publicity and training videos understandable by employees in different countries.
3. Simple operation: just upload the video, select the desired language, and the rest of the translation and dubbing work will be automatically completed by AI.
4. Support a variety of video platforms: directly copy and paste video links on YouTube, TikTok, Twitter and other platforms for dubbing.
5. Maintain the original sound style: When translating, the AI will try to maintain the original tone and style of the characters in the video.
6. Suitable for multi-speaker video: Able to handle conversations with multiple characters, ensuring that everyone's voice is correctly recognized and translated.
7. Fully automated process: From uploading video to translation and dubbing, the whole process is automatic without manual intervention.
8. Multi-language support: Support translating videos into 29 different languages, covering most major languages in the world.
9. Multiple file format support: not only support video files, but also handle audio files, such as MP3, MP4, etc.
(I'm sorting out and publishing for tech freaks)