IT Times reporter Pan Shaoying
When there is friction between folk music and AI, what kind of sparks will collide?
"I thought that AI would be more whimsical when it was written, but getting the score to rehearse felt very much in line with Zhongruan's temperament. We played quite smoothly, and it was easy for us to create a second time, giving us more room to play. Tang Yiwen, a virtuoso of the Shanghai Chinese Orchestra, said.
According to Zhao Zhen, the principal of the sheng department of the Shanghai Chinese Orchestra, AI is not affected by the inherent framework of traditional composition techniques, and the creation of TA may be more free, so it can feel some unconventional expressions in the music, such as rhythmic patterns or melodies that have not been touched before, which also brings a lot of inspiration to musical expression.
As one of the major performances of the 39th Shanghai Spring International Music Festival, on April 13, the country's first AI-composed folk music concert "Zero•One|Chinese Color" co-sponsored by the Shanghai Chinese Orchestra, Tencent Music Entertainment Group (TME) and 1862 Fashion Art Center was held.
With the assistance of AI composition technology, how can artists explore the expression of music together with technology? The second creation based on AI composition requires artists to interact with AI to continuously polish their works, optimize and improve the structure and level of music, and the emotion and temperature of the music, so as to enrich the artistic conception of the concert theme "Chinese color".
"Using the poems of the Tang, Song and Yuan dynasties, combined with the characteristics of Chinese national musical instruments, and allowing AI to generate rhyme, the artist endowed the data with human emotion and vitality. An audience member told the IT Times.
AI generates the national style Yale "Chinese color" in the world
The orient is white, pine, red, lily, and light purple...... These poetic and graceful Chinese colors are heard by the audience in a musical way through the cross-border cooperation between Chinese music and AI.
"The phase and the pillow are in the boat, and I don't know the whiteness of the East. This is from the Song Dynasty poet Su Shi's "Red Cliff Fu" verses, but also the first piece of the concert, Zhao Zhen joined hands with Li Chunxu, Wang Yang and other performers, in the sheng song and the audience to start a continuous dialogue across culture, time and space, a song is played, the aftermath of the sound, the audience has a "sun rises and dawn, the oriental wind comes full of spring" of the long artistic conception.
Subsequently, the "Songhua" performed by the Xindi and Konghou ensemble, the guzheng song "Jiaohong", the Ruan song "Hanlian", the erhu song "Light Purple", and the pipa song "Kneading Blue" appeared successively, and the ups and downs of natural colors are ever-changing, allowing the audience to feel the colorful national rhyme of different colors from the music.
The performance of the suona song "Brown" and the Chinese drum "Xuantian" brought the atmosphere of the whole performance to a climax, and the loud sound of the suona can play a delicate and euphemistic tune against the backdrop of modern musical instruments, which made the audience fascinated; the Chinese drum was sometimes passionate and sometimes solemn, which aroused the enthusiasm of the audience.
It is worth mentioning that the performance breaks through the traditional mirror frame stage, and the orchestra shows the glass curtain wall behind the stage according to the characteristics of the venue, and the glass mirror reflects the figure of the performers and the audience sharing Chinese music at night.
It is understood that the cooperation of this concert originates from the Shanghai Chinese Orchestra's exploration of traditional folk music to seek innovation and change. In February 2024, the Shanghai Chinese Orchestra reached a strategic cooperation with TME.
"AI is a new vehicle to satisfy our curiosity and express the imagination of artists. "It is particularly suitable to use music to express the emotions behind the colors, the ancients gave such beautiful names to colors, inspiring our perception of beauty, behind which there is an intuitive expression of vigorous vitality, as well as the Chinese's cognition and understanding of nature and the universe." ”
"Chinese music is the code of flow, and national color is a solidified culture. In the view of Liu Guoliang, an expert in music archaeology, AI learns traditional culture very quickly, and has no "discriminating heart" for cultural traditions. ”
The large model of qin music breaks through the boundaries of creation
"From QQ Music's listening and humming recognition, to the intelligent sound repair and AI singing of the national karaoke, to the full-stack capabilities of the 'qin music model' such as composition, arrangement and mixing, these successful experiences and technologies have helped us better complete this cooperation. Zhou Wenjiang, vice president of QQ Music Technology and head of Tencent Music Tianqin Lab, said.
Based on the input of Chinese and English keywords, descriptive sentences or audio, the "Qin Music Model" can provide the general public with the ability to "directly generate audio" - text generative music, and can also provide musicians with post-editing needs with the ability to "generate audio and score at the same time" - score generation. The results of large-scale double-blind listening test show that the multi-dimensional subjective score of the "piano music model" in the pure music generation task is stable and exceeds the musicgen 3.3B stereo in the industry.
However, the "test paper" of "Zero•One|Chinese Color" still brought a lot of challenges to Tianqin Laboratory. "The core challenge is to understand and create a wide variety of musical genres. In response to this challenge, the team mainly shared three aspects: type annotation, music understanding, and custom generation. The basic elements of music including beat, timbre, chords, etc., are learned from the massive music data, and the music comprehension ability is an important basic ability of the model. A person from the joint team of Tianqin Lab and Tencent AI Lab said.
After training and fine tuning, a large model of folk music attributes was finally obtained, and on this basis, the creation and generation of 9 folk music pieces were completed. The performers of the Shanghai Chinese Orchestra immediately began to create and rehearse intensively, constantly bursting out of inspiration and broadening their imagination in the second creation, combining their previous performance experience, studying the different expressions of musical instruments, and finally showing the richer musical emotion and artistic appeal of "Chinese color" in the live performance.
Taking suona as an example, in the model training stage, the Shanghai Chinese Orchestra provided suona to play jazz, rock and other materials, hoping to extend the possibility of this instrument through AI, Hu Chenyun, a suona performer and director of the creation planning department of the Shanghai Chinese Orchestra, said: "We do not set limits on national musical instruments, nor do we want to set limits on the creation of AI, but explore with an open attitude." "The final suona song "Brown" is a perfect fusion of classical and modern, national and world.
AI has become a vehicle for musicians' imagination
In fact, the application of AI in the field of music is also showing disruptive development. At present, AI has gradually moved from technical assistance to the stage of music creation. On the one hand, AI will improve the creative efficiency of music creators and use technology to obtain creative inspiration, and on the other hand, AI will also become an important carrier for the realization of human imagination, helping more people to complete music creation through AI, realize their pursuit of music and emotional expression, and present it in a more integrated and innovative form.
TME's own copyrighted songs, with a large number of professional song tags, jointly realize the base part of the large model, and at the same time, after fine-tuning the folk music algorithm and internal and external evaluation, it will finally be launched on TME's music production service platform "Daystar Platform" open to music creators in the whole industry.
Zhou Wenjiang, Vice President of QQ Music Technology and Head of Tencent Music Tianqin Lab, said: "Zero • One Chinese Color presents the collision and integration of cutting-edge technology and traditional folk music, marking a milestone breakthrough in the national music + AI model from 0 to 1. 2024 will be the first year of co-creation between Chinese music and AIGC, and the collision of AI technology and Chinese music will bring more professional and high-quality music content to the industry and the public. ”
It is reported that on April 27, "Zero One|Chinese Color" will be broadcast on QQ Music and QQ Music Video Accounts, and the Shanghai Chinese Orchestra Zone will also be launched simultaneously.
According to Liu Xiankai, vice president of Tencent Music Entertainment Group, AI can not only improve the creative efficiency of music creators and use technology to obtain creative inspiration, but also become an important carrier for the realization of human imagination, helping more people to complete music creation through AI, realize their pursuit of music and emotional expression, and present it in a more integrated and innovative form. The collaboration with the Shanghai Chinese Orchestra represents that with the empowerment of science and technology, traditional folk music can break through the original creative boundaries and present the creativity and vitality of this era.
In the future, the "Qin Music Model" will continue to refine the practicality and efficient creative channel of "text into music", refine and fit rich and diverse song styles and performance scenes, and create more new works of Chinese folk music.