How big is the gap between AI and art?
A few years ago, we laughed at the fact that AI would write inexplicable articles, but in just a few years, AI's progress in the field of art may be beyond the imagination of many people: in the case of the recent OpenAI DALL-E2 image generation algorithm that has been popularized around the world, who would not be surprised by these fanciful but almost human paintings?
THE "MASTERPIECE" OF DALL-E2
Recently, Discovery Quest Records launched a special documentary called "China in The Numbers", which presents the audience with the profound changes in digital technology since the founding of the People's Republic of China 70 years ago. While lamenting the rapid development of the motherland, the leading practical application of interviewING AI (artificial intelligence) in many industries in China has also become one of the "highlight moments" of the entire documentary.
In the documentary, the PDM (Predictive Model) technology pioneered and developed by the QQ music team can not only understand the song, but also intelligently predict the popular potential of the song, successfully help those who are still in the state of zero playback, or even just the potential of demo fragments, the good songs are officially paid attention to by professionals, and have the opportunity to be heard by more listeners until "out of the circle" - from the perspective of artistic creation, although AI painting is far away from us, this PDM good song exploration technology about music. It is already truly changing the music life of hundreds of millions of users and promoting the positive development of the entire music industry.
"For musicians"
How many steps does it take for a song to go from online to popular on the Internet?
For ordinary music fans, the popular path of the song is like an impenetrable "black box", the principle of which is elusive; but in the eyes of practitioners, all the blockbusters have traces to follow.
In the past, new songs would be manually screened by professional music editors from many parties on the music platform, and then matched different promotion resources and went to the public according to various factors such as style and audience. But the data shows that millions of songs are uploaded to QQ Music every year. If you calculate the average song of 3 minutes, it will take decades to manually listen to these songs completely.
The amount of work that is almost impossible for human beings to achieve, and the probability of a good song being discovered, can be described as negligible. "Because a large number of works are online every day, manpower cannot screen all the works, and most of the content does not have the opportunity to be heard." That's the pain point for the entire industry. Ben, PDM Technical Director from QQ Music, said.
But what if AI is used instead of humans?
"We can use AI to listen to it first, and then select good works for human editors to listen to, which can improve efficiency and allow more musicians and works to have exposure opportunities." When it comes to the original intention of the PDM development project, Ben started from the perspective of a musician. "The idea, though, was almost stillborn. Many professional music editors have questioned whether AI can appreciate music, and even if this disruptive technology can eventually be made, it will take a huge investment and unpredictable time, and the team is full of anxiety and worry. ”
In the end, the potential demand for musicians overwhelmed everything. "If we can make this technology (PDM), it will actually be subversive for the entire industry, not only to promote the progress of the entire industry, but most importantly, to help talented musicians get the exposure they deserve." What's more, who has the responsibility and resources to make it happen? ”
Although they were not defeated by the immediate difficulties, this group of technicians still paid a huge amount of effort for PDM. "In the summer of 20 years, we didn't break through at all for a month or two. At this time, it can only comfort the team, which is the darkness before dawn. Ben said. Because no one has done it before, no one can say with certainty that this road will not work, and it is not too much to say that it is a tough one. ”
At the same time of technical research, several algorithm technology code farmers of the team also traveled all over China, continued to visit dozens of music labels and record companies, and directly contacted more musicians through the QQ music campus activities and Tencent musician activities sponsored by the company, and deeply understood the pain points and needs of various production links in the music industry to achieve this technology.
"We didn't know how long it would take, but it had to be made so that excellent but temporarily unknown musical works and musicians would no longer be buried, and it was with this belief that we insisted." Ben remembers the difficulties of the time vividly.
After painstaking research and development, this unprecedented unique algorithm for music has finally come out. "In layman's terms, PDM can predict the explosive potential and quality of a song based on the audio and lyrics of a song." Ben introduced.
Through the AI algorithm, PDM will analyze the melody, chords, arrangement, production, singer's voice, singer's vocals, etc. of a song, and select songs with pop potential. Even for cold-start music works with 0 plays, AI can give full and matched audience recognition and push exposure based on audio and lyric content, so that songs can be more efficiently and accurately covered to suitable target groups. In addition, PDM covers the whole process of song creation and distribution, providing strong technical support for the whole process.
Equivalent to the "AI Bole" of musicians, PDM can select "Maxima" from thousands of songs, and accurately send excellent musical works to people who like it and appreciate it through the recommendation system.
However, can this nascent AI really make everyone love?
Music black technology that can go out even around
"Music is too subjective, how can AI judge the quality of a song like a human brain?" Looking at the PDM in front of her, Ying Jiali, a Tencent musician invited to participate in the technical test, asked the hearts of many people - can PDM really replace manual screening?
"We have regular internal comparison checks." Ben introduced. High-quality songs usually because of their own popularity to harvest more company promotion resources, the technical team will randomly select PDM to give high scores to the human editor for secondary judgment, and finally the high score songs are recognized by the manual editor as high as 90%, and its high efficiency and accuracy have brought great confidence to the company.
In addition to accuracy, PDM technology has also helped a large number of songs and musicians become popular. Public data shows that since the launch of PDM, more than 1.8 million original works of Tencent musicians alone have been "listened to", and 50,000 excellent works and 18,000 independent musicians have been selected from them for traffic tilt. "National-level" hits such as "Wei Wei", "Convergence", "Brazen", "Forgetting the Other Side of the River", "I Am Not Andy Lau" and other "national-level" hits, as well as "Time Sampling", which set a record for the first day of play of independent musicians' streaming songs, have benefited from the "pick" of PDM technology. Young musicians such as Su Han and Bomb Bill have multiplied the number of fans under the recommendation of PDM.
In Discovery's documentary, Tencent musician Ying Jiali, who participated in the test, also witnessed the magic of this technology. At the time, she brought two new titles, "Journey to the Sun" and "Come Up the Hill," for PDM grading. "At first, I thought that 'Journey to the Sun' would not score higher than 'Come Up the Hill' because the former carried more of my personal imprint, while 'Come Up the Hill' was an episode I composed for the film based on the proposition." Ying Jiali said in an interview.
However, PDM unexpectedly determined that the melody of "Journey to Sunday" is more personal and catchy, and it is more suitable for promotion to the audience. She was very surprised by this, "PDM seems to know the mood and subtle thinking of the singer when composing, and in judging the emotion of the song, the score of "God Sunday" is much higher, which is very consistent with my own emotional feelings." ”
In addition to tapping into high-quality music and the "unique insight" of creators, PDM can also become a "highway" connecting music fans and the vast music world - it can find high-quality but not yet well-known niche songs, and push them to fans who like to match, so that fans can not only find what they love, but also broaden their musical horizons in the process of recommendation, which can be described as "killing two birds with one stone".
"The use of PDM doesn't just affect musicians and fans." Ben introduced. "PDM will also promote the development of the whole industry upstream of the music industry, and has a wide range of application scenarios."
"First of all, record companies and labels are also through manpower, experience to select songs, we PDM can provide song evaluation, singer selection, audience portrait analysis, promotion planning and other aspects of the upstream of the industry to provide song evaluation, singer selection, audience portrait analysis, promotion planning and other aspects of the advice, covering the whole process of song creation, distribution, so that practitioners can quickly judge the fit of the song and the market, and match the song with the right resources, find the direction of promotion, and greatly save the cost of the record company." Ben believes that the convenience that PDM brings to record companies will also feed back to musicians, "assuming that every record company uses PDM, the most talented people will quickly stand out, the quality of music creation will be much higher, and high-quality creation will also feed upstream, eventually forming a benign closed loop." ”
"PDM also has many application scenarios, which are also very exciting." Ben mentioned the team's newly developed MMatch technology. "MMatch technology is a public music solution for offline businesses, which can match the appropriate music for offline public broadcast scenes such as physical stores, cafes, convenience stores, etc., and PDM can first screen out relatively better and more matching scene songs, on the one hand, it can make the public broadcast scene have more diversified music choices, on the other hand, it also opens up a broader traffic entrance for musicians."
In addition to the diverse application prospects, this technology has also received many recognitions. At the International Audio Retrieval and Evaluation Competition (MIREX), the most authoritative competition recognized by the international computer music academia, the QQ Music (PDM) technical team achieved breakthrough results in "Patterns for Prediction", breaking the world record in one fell swoop and receiving widespread attention from the industry.
It is not difficult to find that technological innovation is changing our musical life. In the past, looking for songs to rely on friends to pass on, magazine recommendations, and even "opening blind boxes" in audio and video stores to try their luck, today there are AI agents to do it; and the tapes and CDs that were previously appreciated have also evolved into "digital collections" in the QQ music collection Luo Dayou and Tengger.
In addition to the innovation of listening to songs, a more immersive experience and participation in the future is also approaching, virtual entertainment, meta-universe is no longer far away. In addition to the previous virtual island TMELAND, these two days of search QQ music also saw a new feature "VR album" room - it should be a technology they are also testing, so that fans can enjoy the artist's album in the room with a virtual image, and can control the virtual person to make a variety of actions, chat with the same netizens, etc. - enter the room can see a variety of virtual images rubbing shoulder to shoulder, everyone discusses with each other, listens to songs, and its enthusiasm is no less than an offline concert.
VR album, another new form of entertainment that breaks the "offline anxiety"
What will tomorrow be? Based on the scientific and technological innovations that have been realized so far, can we boldly predict that high-quality music creation will no longer be overwhelmed by the tide of traffic, fans can hear the good songs they like, have a richer music play and experience, and the entire industry can move forward in a more efficient way.
This may be enough for us, and for the hard-working technologists behind music technology.
Maybe you can still consider joining it?