Reporting by XinZhiyuan
Edit: Good sleepy David
【New Zhiyuan Introduction】AI has been hot for so many years, but the framework is still "two-person turn", is the domestic framework really a can play nothing?
In late 2020, the second generation of deep learning neural networks developed by DeepMind shocked the structural biology community.
The advent of AlphaFold 2 solves the protein folding problem that has plagued scientists for decades.
Just last month, a team from China successfully completed predictions about the protein's structure.
The optimized model has improved the performance of single-step iteration by 40%, the TM-score has reached 85 points, comparable to AlphaFold 2, and more importantly, the code is also open source.
Prior to this, another team of joint teams had also optimized for AlphaFold 2, which increased the inference efficiency of the model by 2-3 times year-on-year.
An important factor in the researchers' ability to make such breakthroughs in protein structure prediction is the AI framework used by the model, MindSpore.
As a Ascend Mind MindSpore that was only open sourced in March 2020, its popularity has reached the first place in the Gite list, and there are more than 300 open source models based on this domestic AI framework.
What is an AI framework?
To understand why MindSpore is so popular, you first need to understand "what is an AI framework" .
As we all know, the three pillars of artificial intelligence are data, algorithms, and computing power.
The AI framework is a set of standard interfaces, feature libraries and toolkits for the design, training and verification of algorithm models.
During the development process, the AI framework is responsible for providing developers with mathematical operations to build neural network models, converting complex mathematical expressions into computer-recognizable computational graphs, and automatically training neural networks.
The resulting model can be used to solve the problems of classification and regression in machine learning, and to achieve application scenarios such as target classification and speech recognition.
Source: China Academy of Information and Communications Technology
In addition to completing the engineering implementation of AI algorithms, the AI framework can also greatly improve the learning efficiency of artificial intelligence, strengthen the model capabilities of AI algorithms, and play a role in carrying forward the upper and lower levels.
Downward, the underlying hardware computing resources can be invoked, which can mask the underlying differences and provide good execution performance; upwards, it can support the construction of AI application algorithm models and provide a standard environment for algorithm engineering implementation.
Having talked about so many theories, it is practical to understand it well with one example.
TensorFlow and PyTorch, as we know it, are the most famous and most used AI frameworks at this stage. They were open sourced in 2015 and 2016.
Within 1 year, 4 large models were issued in a row
In comparison, Ascend Mind MindSpore, which was only open sourced in March 2020, can be said to be quite late.
However, the "latecomer advantage" also allows Ascend MindSpore to get the ability to natively support large models.
In terms of design, Ascend MindSpore adopts a functional differentiatable programmable architecture, supports all-scenario collaboration, and provides Python programming paradigm to make AI programming simpler. In addition, Ascend MindSpore also unifies the encoding of dynamic and static diagrams, and the coding method of stand-alone and distributed training.
Ascend MindSpore overall architecture
For large models, the efficiency of parallel training and the ability to support the model structure are extremely important indicators.
Ascend MindSpore offers a wealth of parallel strategies: data slice preprocessing, data parallelism, operator parallelism, optimizer parallelism, pipeline parallelism, MoE parallelism, and multi-replica parallelism. Its ability to automate parallelism has reduced the amount of parallel code by 80% and tuning time by 60%.
In contrast, the parallelism capabilities of the PyTorch/Megatron framework can only support the Transformer model structure at present, while TensorFlow supports a lack of parallelism strategies and relatively few developers.
In terms of generalization of model structure, Ascend MindSpore provides all-round support for dense Transformer, sparse MoE+Transformer, convolution, convolution +Transformer, high-dimensional sparse, etc. Compared to other frameworks in the industry, the support model structure is the most complete.
With the ability to natively support large models, 4 large models based on MindSpore have now been released, and 2 of them have begun to be industrialized.
In May 2021, the technical team led by Pengcheng Laboratory released the world's first 200 billion dense parameter Chinese NLP model "Pengcheng. Pangu realizes large-scale distributed training on the 2048 card hashrate cluster through ascendant MindSpore's hybrid automatic parallel mode.
"Pengcheng. Pangu is superior to the SOTA model in most of the 16 downstream tasks, with 11 zero-sample learning tasks leading, single-sample learning tasks 12 tasks leading, and small sample learning tasks leading 13 tasks.
In September 2021, Pengcheng Lab released the "Pengcheng. Shennong", which includes protein structure prediction, small molecule generation, target and small molecule interaction prediction, and new antibacterial polypeptide design and effect evaluation.
Also in September 2021, the Institute of Automation of the Chinese Academy of Sciences and the MindSpore community jointly released the world's first large model of 100 billion parameters of graph, text and sound, "Zidong. Too early".
It has both cross-modal understanding and generation capabilities, and can lead the current industry's SOTA models in terms of graphic and text cross-modal understanding and generation performance, and efficiently complete downstream tasks such as cross-modal detection, visual questioning and answering, and semantic description.
At two international competitions in 2021, ACM Multimedia and ICCV, "Zidong. Taichu" all won the first place.
In addition, the world's first special framework for intelligent interpretation of remote sensing images released in December 2021, "Wuhan. LuojiaNet", also based on the Ascend MindSpore framework.
Tian Kunyang, product director of Huawei's Ascend Computing business, said that in addition to the four big models that have been released, there are more than a dozen under development. Ascend MindSpore uses the method of compiling small models with large models to make generalization bigger and be able to cover more scenes.
From 0 to the domestic lead, it took only 2 years
However, whether an AI framework can become mainstream, in addition to these "majestic" models, largely depends on rich open source projects and a large developer base.
At this point, Ascend MindSpore's results are quite impressive.
On China's localized code hosting service platform Gitee, 22 repositories have been established, with a total number of stars reaching nearly 16,000, of which the main position Ofte Index is 89 points, leading similar projects and ranking first in artificial intelligence projects.
In an interview with Xinzhiyuan, Nakata Kunyang said that in order to build a developer community, In addition to always committing to feature optimization and ease of use, MindSpore is also focusing on creating a learning and growth environment where the community and developers can grow together.
First of all, the community has designed a perfect learning path for different developers, providing rich learning and growth resources through community activities, community documents and cases, technical certifications, awards, etc.
Secondly, the community always adheres to the principle of putting developers first, the development process is open and transparent, and all developers who have contributed to the version issue certificates to them, so that developers can deeply participate in the growth of the community.
Finally, the community has established a rich community of different technical directions, and has a variety of roles of organizers, evangelists, and developers, so that every member of the community can find the most suitable entry point.
For now, the idea is still very popular. As of March 2022, Ascend MindSpore's cumulative downloads have exceeded 1.42 million, the number of developers has exceeded 640,000, and the number of community contributors has exceeded 4,000.
The number of papers at the top meeting once surpassed TensorFlow
Not only that, but the academic community also welcomes this domestic AI framework. At present, more than 120 research institutes and universities have used Ascend MindSpore.
According to Paper With Code, in the fourth quarter of 2021, there were 220 papers based on MindSpore, accounting for 6% of the total, ranking third. In October 2021, it accounted for 11% of all AI frameworks, ranking second, behind PyTorch.
The total number of papers with Ascend MindSpore as the AI framework last year exceeded 300.
Source: Paper With Code
According to Tian Kunyang, at present, the undergraduate graduation thesis of the School of Computer Science of Wuhan University of Technology has begun to be implemented using the MindSpore framework. Students also reported that with the iteration of the version update, the ease of use of the MindSpore software has taken a qualitative leap.
In the competition of the major top clubs, the appearance rate of MindSpore is also very high.
Professor Jiao Licheng's team won the 2021 IEEE GRSS Data Fusion Contest Track DSE championship with Sheng si MindSpore as the framework, and won a total of 11 first and second prizes in the ICCV 2021 competition.
In addition, the team of Professor Yang Yang of Nanjing University of Science and Technology won the SIGSPATIAL 2021 GISCUP International Champion, and the MARS_WHU team led by Professor Du Bo and Professor Ye Mang of Wuhan University won the ICCV 2021 MMVRAC Track Championship.
However, compared to PyTorch, which accounts for more than 70% of all major summits, Sheng Si MindSpore still has a long way to go.
In order to help developers and researchers better solve the difficulties or questions encountered in the development process, Ascend MindSpore has established a "rapid response mechanism", which sets up a corresponding group for problems in the community, clarifies the SLA, and responds within 1 hour.
Recently, in the "I grew up with MindSpore" two-year celebration campaign initiated by the Ascend MindSpore open source community, more than 100 developers told their own small stories.
From these sharing, we can actually see that everyone still likes the Ascend MindSpore framework that is constantly improving.
Eric Steven Raymond, a pioneer of the well-known open source movement, once said, "Given enough eyeballs, all bugs are shallow."
Open source for two years, has been stunning MindSpore AI framework, how many "eyes" from the three worlds of production, learning and research will attract in the future? How many big models will be born that will disrupt industrial trends and change the future computing landscape?
Open source for two years, Ascend MindSpore from scratch to achieve "domestic leading", in the future, with more developers eagerly concerned about the "eyes", these big models will build a "digital bridge" between the laboratory and the factory?
Hopefully, the moment the future answer is revealed, there will be light in all eyes.
This answer, I believe That MindSpore will not make us wait too long.