The index finger of the left hand is extended horizontally; the right hand extends its thumb and little finger, crashing into the left index finger...
Sign Language: Sprint
"Wu Dajing's last sprint!" In the last bend, Wu Dajing took the lead in rushing out of the bend and crossing the finish line! ”
During the central video competition, the central video AI sign language interpreter listened, bringing sign language commentary to the moment of winning the gold last night.
At present, there are 27.8 million people with hearing impairments on the mainland.
The sign language host in the lower right corner of the TV screen is the "narrator" in their eyes.
At the same time, for the hearing impaired, there is a tencent 3D sign language digital person Xiao Cong, who will launch Tencent Sports.
Xiao Cong's sign language commentary "China wins the first gold"
Xiaocong and Tingyu are both created by Tencent's technical team and are characterized by:
Sign language skills are close to those of real people;
The so-called endless learning, they can learn on their own, quickly supplement a large number of new words, hot words.
In recent times, Lingyu and Xiaocong will provide sign language commentary services for the broadcast interviews of ice and snow events.
27.8 million people need sign language moderators
How to make TV better for the 27.8 million people with hearing impairments?
The answer is, sign language host!
But sign language explanation is not simple.
Let's first look at the word order problem -
The normal word order arrangement of Chinese and sign language is not the same. Let's look at an example -
Chinese: Cat chases mouse
Sign language: Cat and Mouse Chase
This adds difficulty to sign language explanations.
If you follow the normal chinese word order, then the sign language explanations seen by the hearing impaired are actually out of order.
In addition to the problem of word order, sign language has another feature: there are no imaginary words and quantifiers.
Take two examples to feel the following -
Chinese: Two books
Sign Language: Book II
Chinese: Heavy snow
Sign Language: Snow + Degree of substantial body swing embodiment
Imagine that inappropriate omissions can lead to a thousand differences in the understanding of meanings.
For example, "two bundles of books" might be understood as "two books."
In addition, sign language has another feature:
It is not only necessary to express information such as gestures, but also rely on facial expressions and body movements.
The survey found that the vast majority of people with hearing impairments can only understand less than 60% of sign language news.
In addition to the difficulty of sign language translation, another obvious status quo is -
There are too few sign language hosts.
Therefore, it is very necessary to develop AI sign language hosts with accurate translations.
In October 2021, the State Administration of Radio, Film and Television clearly pointed out for the first time in the "14th Five-Year Plan for the Development of Science and Technology in Radio, Television and Online Audio-Visual" that it is necessary to promote the wide application of virtual anchors and animation sign language in the production of news broadcasts, weather forecasts, variety science and education, etc., innovate program forms, and improve the efficiency and intelligence of production and broadcasting.
Let digital people master professional sign language interpreters
The question is, how do digital people learn professional sign language interpreters?
Behind The listening and xiaocong, it integrates 3D digital human modeling, speech recognition, semantic understanding, machine translation and image rendering technology.
Speaking of machine translation, our programmers also nibbled on the National General Sign Language Dictionary.
After a long period of sign language research and team building of sign language consultants, we have developed a sign language translation system-
Simply enter the listener language to generate highly accurate sign language representations with low latency through machine translation, such as:
Input: He is my sign language teacher
Pre-treatment: He is my sign language teacher
Translator: He I Sign Language Teacher Yes
Then, drive the sign language digital people to accurately express -
Based on Tencent's multimodal end-to-end generation model, joint modeling and prediction generate sequences such as actions, expressions, and lip movements with high accuracy, so as to achieve natural, professional, and easy-to-understand sign language effects.
Through this technology, AI sign language can be understood by more than 90%.
All along, Tencent has been actively advocating that the whole society pay attention to information accessibility, and we will open up our technology and capabilities to the society.
Looking forward to the future, there will be more AI sign language hosts online in the TV column.