Cloud computing, a new generation of IT infrastructure; graph computing, is a technology that completely depicts, calculates and analyzes the relationship between any thing and thing in the objective world.
"The cloud calculates the world, and the future is created", in the era of digital intelligence, the cloud is the base and the map is used, and the two complement each other. This is probably also the profound meaning of Huazhong University of Science and Technology's Yuntu team participating in the 7th China International "Internet +" College Students Innovation and Entrepreneurship Competition with "Yuntu" as the team name.
One minute on stage, ten years offstage
The 7th China International "Internet +" College Students Innovation and Entrepreneurship Competition recently ended in Nanchang, Jiangxi Province, this year's new industrial proposition track has become the focus of attention, and finally, the Yuntu team of Huazhong University of Science and Technology and the Chenglian Zhitu team of South China University of Technology stood out from the 50 teams that advanced to the final and won the track gold award.
When Zhao Jin, a 2017 doctor of the Key Laboratory of Service Computing Technology and Systems of Huazhong University of Science and Technology and his team's brothers, stood on the podium, how many sleepless nights since participating in the competition, the monotonous life of dormitories, canteens and laboratories, from the school competition to the provincial competition to the national competition, all turned into a rising mouth and a relieved smile. All the effort and effort was worth it!
The partners imitated the scene of the live interview and let Zhao Jin give an acceptance speech, he smiled shyly and said the voice of the whole team: "The experience of participating in this competition is unforgettable for me. In the early stage of doctoral studies, my main energy is focused on scientific research, taking the opportunity of this competition, I can combine our research content for many years with the needs of enterprises, effectively solve practical business problems, and make us feel that the scientific research work we have done before is very meaningful. ”
Representative of the Yuntu team of Huazhong University of Science and Technology
From the initial school competition to the provincial competition, and then to the final national competition, the Yuntu team invested a lot of time and energy from polishing the content of the entries, making PPT, participating in roadshows, and defending. No wonder Zhao Jin sighed: "One minute on stage, ten years off stage." ”
Another backbone of the Yuntu team, Yu Hui, a 2020 doctoral student in charge of system software research, also felt the same way: "The feeling of the whole competition is that time is tight and the task is heavy. The cohesion of the team members, the care and help from the school teachers, and the support of Huawei technology experts make us feel the great energy of unity and cooperation. In particular, our instructor, Associate Professor Zhang Yu, is hands-on in everything, and every polishing meeting and every roadshow rehearsal is personally present, leading the team to continuously polish and optimize the project. And when any problems arise, he leads us to find ways to solve them at the first time, and then constantly revises and improves. Special thanks to the hard work of guiding Teacher Zhang Yu! Although the 10-minute speech of the final roadshow was completed by his brother Zhao Jin alone, behind him, there is the support, effort and dedication of our entire team. ”
"We were able to win the final gold medal, which is well deserved and the best reward for this team." When he said this, Afterglow's tone was full of confidence, pride and gratitude.
This gold award is well deserved
You must be wondering, how much of this gold medal won by the Yuntu team this time? China International "Internet +" College Students Innovation and Entrepreneurship Competition is a global competition, this year is the seventh edition, the participating teams from 121 countries and regions at home and abroad 4347 colleges and universities, the total number of applicants reached 9.56 million. The Yuntu team finally won the gold medal, which is really five levels and six generals.
Looking at this work of the Yuntu team, relying on the proposition of "Kunpeng BoostKit Big Data Spark Graph Algorithm Optimization" selected by Huawei Kunpeng's underlying technology, it has realized the world's first high-performance graph computing system for concurrent graph analysis tasks, and has achieved multi-level improvement in traditional graph computing performance by efficiently using the many-core architecture and hardware acceleration unit of Kunpeng processors, adapting to Euler's open source operating system.
The team led by Zhao Jin consists of 5 people, including doctoral, master's and undergraduate students. Everyone divides labor and cooperation, Zhao Jin is mainly responsible for the technical and framework design of the entire project, others are responsible for software development, some are responsible for engineering development, and some are responsible for making and optimizing PPT. Although this is an improvised team, and there is almost no experience in participating in similar competitions before, this does not affect the tacit understanding between the team members, but can stimulate their enthusiasm and talent. According to Zhao Jin, although the team members perform their respective duties, all of them have one thing in common, that is, they have been engaged in research related to graph computing before, the longest of which has been ten years. Under the leadership of Associate Professor Zhang Yu, the team has made a number of key technological breakthroughs in the field of graph computing, developed the world's first concurrent graph analysis task system, and the world's first asynchronous graph computing accelerator based on many-core processors, and the research level has reached the international leading.
How good is Huazhong University of Science and Technology's research on graph computing? At the 2021 Graph Computing Challenge, one of the most influential international competitions in the field of graph computing held in August this year, the graph computing team of Huazhong University of Science and Technology won the global championship. This is the first time that China has won the championship in the competition. Awesome! In this way, the Cloud Atlas team can win the gold medal of the competition with its graph calculation works, which is the best inheritance and the expectation of all.
"The first time I saw this proposition of the competition industry track, the first feeling was that it was very suitable for our preliminary research work, and at that time there was an impulse to try, hoping to organically combine the basic research work of graph calculation that I did with the actual needs of the enterprise." From Zhao Jin's tone, you can feel this urgent desire, "In the past, we have also cooperated with some companies and the results have landed, but this time based on huawei's Kunpeng platform is still the first time, and I also want to challenge myself." ”
As a member of the team, Yang Yun, who is studying in the second year of graduate school, has a lot of feelings: "Through this competition, I deeply feel the obvious difference between industry and academia. Our principal once said, 'Not only should the papers be published in top journals, but also in the land of the motherland'. Through this competition, I have a deeper understanding and understanding of the phrase 'publish the paper on the land of the motherland'. The purpose of our research is to promote the industrialization of new technologies. Huawei's proposition highly integrates scientific knowledge with industry practice, which has benefited us a lot. ”
The new industrial track added this year aims to promote the close connection between the intellectual, technical and project resources of universities and the needs of economic and social development. Huawei has been deeply involved in the industrial proposition track, providing a total of 32 innovative proposition projects, attracting 1,205 teams from 337 universities and nearly 10,000 students to participate. Another Chenglian Zhitu team that won the gold award is based on the proposition of "using MindSpore to design and train image style migration models" selected by Huawei Ascend AI technology capabilities, and innovatively proposes a high-performance image style migration algorithm and gives realistic application scenarios.
Whether it is the previous research or the current fame through the competition, the original intention of the Yuntu team has not changed, that is, to make a world-leading graph computing system, promoted to Chinese enterprise users, and landed in multiple actual production scenarios. With the support of Kunpeng BoostKit-related technologies and Huawei's professional guidance, the Yuntu team has taken a substantial step from technical theory to application practice.
There is no obstacle that cannot be turned over Huawei's support is indispensable
For technical research such as graph computing and large-scale concurrent processing, Zhao Jin is extremely confident, and his team's research results are international firsts, and he has published many top papers. However, after getting the competition question, for the map computing system and Kunpeng, Euler and other software and hardware platforms to adapt, especially running on the Huawei Kunpeng multi-core processor, in fact, he has no bottom in his heart.
Graph computing is now a very widely used technology, is a mainstream model of big data processing, is used in education, medical care, military, finance and many other fields. In reality, a lot of data can be expressed as a graph structure, such as social network graphs, road traffic graphs, flight graphs, grid graphs, etc. The core of the "Kunpeng BoostKit Big Data Spark Graph Algorithm Optimization" is how to further improve the accuracy of the graph algorithm while minimizing the processing time.
"Based on the Kunpeng platform, the existing industry's open source algorithm has a processing time of 600 seconds when the accuracy is only 75%. Huawei hopes that when improving the accuracy of the algorithm to 90%, it can control the processing time within 90 seconds. Zhao Jin said, "When we conducted the first test based on the Huawei platform, we found that when the accuracy is 90%, based on the universal open source algorithm, the execution time is 30,000 seconds." However, after our optimization, based on the cloud map system algorithm, when the accuracy reaches 90%, it only takes 13 seconds, and the efficiency is increased by more than 2300 times. ”
For the Yuntu team, how to give full play to the performance and efficiency of many-core processors on the Kunpeng platform is the first threshold to be crossed. In order to efficiently play the role of many-core processors, the Yuntu team proposed a new technical route, that is, to process the graph data in a decoupled manner, so that these cores do not depend on each other as much as possible. Under this principle, they performed a "major operation" on the architecture, so that the computing power of the Kunpeng processor was fully released, laying a solid foundation for the subsequent tuning of the graph computing system.
Zhao Jin also mentioned a detail, in order to further improve the accuracy of the graph algorithm to meet the propositional requirements, it is necessary to deal with the concurrency of 150,000 BFS, and before this, the Cloud Atlas team has never encountered such a huge parallel processing problem. How to get 150,000 graph computing tasks deployed on three machines provided by Huawei and run efficiently? The Cloud Atlas team made special optimizations. The 2300x efficiency increase is already telling the story.
The Kunpeng BoostKit Enablement Kit helps Kunpeng developers and ISV partners accelerate application performance in eight typical application scenarios: big data, distributed storage, database, virtualization, ARM native, Web/CDN, NFV, and HPC. "BoostKit is an important manifestation of Kunpeng's technical capabilities, and it does play an important acceleration role in the process of completing the work. Based on it, we have also developed a graph computing system that is more suitable for the Kunpeng architecture- the cloud graph system, which greatly improves the performance of the graph computing system. This is critical for different common graph architecture application migrations. Zhao Jin said.
"Throughout the competition, we received careful guidance from Huawei experts. For example, when we first got the competition questions, we were not accurate about the data sets, accuracy, and judging standards involved in the questions; in the process of competition, when we encountered some problems, we could not find the corresponding answers in the Kunpeng ecology. At this time, Huawei's experts not only patiently answered and guided us, but also specially organized an online meeting to demonstrate for us one by one. The support from Huawei makes us very grateful and touched. Afterglow said.
Industry-academia interaction goes hand in hand
For the Yuntu team, the entire process of participation, from a technical point of view, there is no obstacle that cannot be turned over, but the commercialization of the project is a "roadblock". Because several people in the Yuntu team who have been engaged in research have no entrepreneurial experience, they have almost no concept of how to write business plans and allocate shares. At this time, it is also Huawei's experts who come forward to guide them and straighten out their thinking. Zhao Jin was able to cope with it freely when he finally replied, and he successfully broke through the barrier.
"Whether in academia or industry, Huawei's technologies and products are at the international leading level." After having the first intimate contact with Huawei Kunpeng through this competition, Zhao Jin's overall feeling is, "The cloud map system we developed can reach the top level in the industry, and it is also thanks to the support of Huawei Kunpeng multi-core processors and so on." In the future, we will develop more software systems, accelerators, etc., and hope to deploy and apply them to Huawei's Kunpeng platform. ”
The Yuntu team has the same original intention as Huawei, and both want to contribute to the development of China's information technology. In the future learning and work, it is also the good wish of the Yuntu team to deploy more technologies and applications to Huawei's self-developed software and hardware platforms.
Ideals have been illuminated into reality.