Dancing machine? Somatosensory games? Smart fitness mirrors? Open this mobile phone application with sound and picture, there is a sense of "one specialty and multiple functions" experience. Originally, a group of teachers and students of shanghai film academy of Shanghai University who could not dance used film special effects technology to capture the various beautiful dance postures of more than a dozen non-genetic people across the country for more than two years, and generated three-dimensional "animation models" through virtual reality, augmented reality and other techniques, developed into "intelligent folk dance" APP, so that mobile phones could be transformed into dance magic mirrors in seconds, and intelligent scoring was scored at the dance forest conference.
This National Arts Fund "Folk Dance Digital Promotion Platform Construction" public welfare project, from square dance aunts to kindergarten children, allows dance learners of different ages to "mirror" at least 20 kinds of dances of various ethnic groups on the Silk Road through mobile phone screens. A few days ago, the Shangda team first launched the application on the Android platform, and has submitted it to the Apple Store, and it is expected that it will be reviewed and put on the shelves next month for the public to download for free.
"Smart Folk Dance" APP interface
【Motion Capture: Authentic Transmission into the Database】
Northern Shaanxi Yangge, Lanzhou Taiping Drum, Mongolian Cup Bowl And Chopstick Dance, Gannan Guozhuang Dance, Tibetan Barang Encouragement, Uyghur Daolang Dance, Hui Stepping Dance... With so many characteristic dances, although there are massive video materials on the Internet, it is difficult to search for small videos to learn online. To this end, teachers and students of the Digital Media Creative Engineering major of the Shanghai Film Academy have a new idea: whether they can be like the virtual fitness coaches on the market, let the folk dance also appear on the mobile phone "stage", for the dancers to follow the "simulation teacher", and introduce a 360-degree full view, which can be arbitrarily scaled local movements.
Non-hereditary heirs and the Shangda team took photos at the special effects center
Where do dance teachers come from? Under the leadership of Teacher Tian Feng, this team of more than 20 people invited the most authentic non-hereditary people along the Silk Road. As a representative project of national intangible cultural heritage, the Ansai waist drum integrates dance, martial arts, gymnastics, percussion, blowing music, folk songs, etc. in the process of development and evolution. Hearing about this platform, Liu Yanhe, the national inheritor of the Ansai waist drum, hit it off with teachers and students in Shanghai, and together with his assistants, came to Shanghai from Yan'an and walked into the shanghai municipal engineering technology center - the film special effects center.
In front of the green background of the lab, they put on "motion suits" and danced for an hour or two. For the project, three digital versions of the Anse waist drum music section were "recorded", including the old version, the youth version and the double version, forming a basic database of tens of megabytes. After capturing the movements of more than a dozen such non-genetic people from all over the world, they collected the folk dance movements of different ethnic groups along the Silk Road and built a digital Silk Road dance gene bank.
【Human body score: the entry threshold is not high】
In the team, cheng Wanqiu, a second-year graduate student, may be the only amateur dance enthusiast. Facing the video capture lens on the screen, she practiced for the first time the Shanghai Municipal Intangible Cultural Heritage - "Playing Lotus Xiang" dance, and the "instructor" was a virtual dancer based on a successor of Jinshan Langxia Primary School. Cheng Wanqiu follows and dances in real time, and follows the rhythm point prompts at the bottom of the screen to make iconic dance moves. Although not very coherent, the platform still scored a "passing score" of more than 60 points. In addition to the praise of the prompt tone, the teachers and students on the side also applauded it.
From the right step, the pad step to the horse step, from the neck movement, shoulder shaking to the trembling knee, from the hard wrist, the soft wrist to the opposite wrist, and the three steps and one stroke, etc., do you master these basic dance movements? Since it is a smart folk dance, it must be able to be a sparring partner and score. At present, the team is still upgrading and iterating, and continues to optimize the artificial intelligence training scoring module.
Artificial intelligence (AI) requires a lot of computing power, but integrating into a small platform like a mobile phone requires the right calculation accuracy to be controlled. Li Yuzhi, a second-year research student in charge of software programming in the team, told reporters that because there is no need for external motion sensing equipment, only the camera is used to detect the physical dynamics of the dancers, so they exclude background interference such as backlighting and multi-color, and try to reduce data interference. Under the blessing of such an algorithm, this human body scoring system is more "tolerant", and the entry threshold for beginners is not high, and several trainings can also get high scores.
[Correction error: hand and foot details are also high imitation]
Some Silk Road dances, such as Dunhuang dance, are not owned by a single ethnic group, and there is no specific successor, so the team invited Zhu Yafei, a professional teacher who graduated from the Beijing Dance Academy, to demonstrate. From the flying shape to the rebound pipa, Teacher Zhu performed it beautifully. However, the motion suit wrapped around the body cannot cover the details of the movements of the hands or feet at the same time.
The project team is developing software
Team graduate student Wang Xuefei and other students who are good at visual image production patiently observe the teacher's body language between the hands and feet, such as the thumb and little finger are often particularly close when the Dunhuang dance hands dance, so they constantly make corrections and adjustments on the data model, so that the digital model forms a high imitation effect on the original dancer.
"Folk dance promotion platforms should emphasize both professionalism and universality, and achieve a balance between inheritance and popularization." Ding Youdong, vice president of the Shanghai Film Academy, told the Jiefang Daily Shangguan News reporter. For example, after communicating with the representative dancers of the Kazakh "black horse", the platform mainly grasps a series of classic movements with the most iconic in the dance, so that people can see and learn, showing the riding posture of this nation on horseback, which is very artistically expressive and suitable for the people to exercise.
It is worth mentioning that the Tian Feng team uses their respective expertise to develop and teach synchronously, and even according to various folk dance rhythms, independently compose music and orchestration in the APP, and independently create dance accompaniment, so that music and dance are harmonious and harmonious, and audio-visual integration is enjoyed.
Column Editor-in-Chief: Xu Ruizhe Text Editor: Xu Ruizhe Title Image Source: Xu Ruizhe Photo
Source: Author: Xu Ruizhe