Anthropomorphizing machines, from "artificial intellectual disability" to "artificial intelligence"

On May 27, Dark Horse Entrepreneurship held the "2023 Leap Jump • Dark Horse AIGC Summit" in Beijing. The theme of the conference is "Envisioning a New World, Building a New Landscape". Justin Cassel, former deputy dean of the School of Computer Science at Carnegie Mellon University and former chairman of the World Economic Forum (WEF) Council for the Future of Computing in Davos, as well as executives from 360 Group, KLCII, Kunlun Wanwei, Yunzhisheng, BlueFocus, Wondershare, and Zhichuangyu attended the event and had in-depth exchanges with thousands of participants.

At the summit, Huang Wei, founder and CEO of Yunzhisheng, shared the theme of "The Road to a Smart Future".

Here's a breakdown of the shared content:

Yunzhisheng is an entrepreneurial veteran who has focused on speech recognition and natural language processing for the past decade, and our technology field and big model are the closest. On May 24, we released the Shanhai Grand Model in Beijing. As a startup, let's take a look at the experience on the road to the evolution of the big model.

At first we wanted to do it the way the experts did, we wanted to give the machine some methodology, and ten years ago, the machine started learning from the wrong feedback. These are the approximate stages and paths in the past AI technology.

Today OpenAI launched ChatGPT and pre-trained models, the whole intelligence became more anthropomorphic, first we read all the text known in the world with very powerful computing power, and trained to form a large model. It is especially like the baby brain, there may be tens of billions, hundreds of billions of parameters, and the human brain is different, the baby at most only inherits the appearance and personality of the parents, etc., but the brain of the large model inherits knowledge, this is only the initial state, and then through fine-tuning and other ways, like children will have various education in the process of growth, the evolution of the entire large model is more anthropomorphic.

This is a change in the whole of artificial intelligence.

What are the essential changes between AGI today and before? Before December 2022, the entire artificial intelligence is still a kind of discriminative artificial intelligence, doing judgment questions, special systems and intelligent modules, and doing some specific tasks. On the one hand, the performance of artificial intelligence is not so intelligent, and it is often criticized by others that "what you provide is artificial intellectual disability", so that the ceiling of artificial intelligence capabilities in the past is low.

Second, in many scenarios, the needs of customers are very different, but the capabilities of artificial intelligence are not so strong, and many companies and teams use various customizations to meet them. AI companies are not like high-tech companies, and in the past decade, only discriminative AI is a manual workshop era. But now with large models and more powerful general-purpose capabilities, artificial intelligence has begun to enter the industrial era.

With new generation capabilities and emergence capabilities, one model can solve different problems in many scenarios. In today's era, the big model of artificial intelligence is the generator, and before the engine was invented, the Middle Eastern countries were not so rich, and the value of oil was not so great. Just like today, data can be turned into fuel and capabilities, and this ability can be used to empower thousands of industries.

Why can Yunzhisheng launch a self-developed large model in a short time?

We were founded in 2012, is the first in China to apply deep learning to speech capabilities, previously seen in science fiction films, in 2012 launched a speech recognition engine based on deep learning, at that time deep learning as the entire technical framework of Yunzhisheng.

Seeing AlphaGo in 2016, we put medical products on the ground in the hospital to help doctors in Peking Union Medical College Hospital and greatly improve work efficiency. In the hospital scenario, just efficiency tools are not enough, the real intelligence of artificial intelligence is cognitive intelligence, Transformer was proposed in 2017, cognitive intelligence behind the need for more powerful computing power.

With these foreshadowings, I have accumulated a lot of experience in both academic and engineering aspects. This experience is your ability to make a living for the individual, but it is the core competency for the company to win in the market. After looking at the ChatGPT framework, we found that none of them were new, they were all existing engineering combinations, and we quickly combined this ability into the development of large models.

Three days ago, we released a large commercial model called Shanhai. Ran through pre-training, instruction fine-tuning, and augmented learning based on human feedback, and saw the long-awaited emergence of capabilities. At that time, the team was wondering if they wanted to give it a name, and I was traveling a lot during that time, and I thought the name was good. The sea is magnificent, there is room for large, reflecting the infinite generation ability of the large model, the mountain is the mountain back, we know what can be said and what cannot be said, which is precisely to emphasize the generation ability of the large model, but also to emphasize the safety compliance of the large model.

There is a very interesting phenomenon, everyone is talking about the big model, and the domestic attention to the big model is after the Spring Festival, but everyone does not talk about this, and they have no bottom in their hearts. Until today, there is a view that this matter can not be done only with technology, even if the people are in place, but the training cost is very large, and it is extremely expensive. The big model is not a scientific revolution, not the invention of new algorithms, it is a combination of existing algorithms to make it bigger, most of them have a price, of course, there are many projects in it. The point is right.

Conversely, if you think that the next 10-20 years big model is a big opportunity, BAT can't invest it, and give up, I think there is still a chance.

In the past few years, Yunzhisheng does not need particularly good scientists, I even think that this matter is not a matter of scientists, scientists have not played so much computing power, and do not know where the scene is, so the result must be bad. Manufacturers with scenarios are the most likely to succeed.

The name of mountains and seas also has a meaning, love across the mountains and seas, mountains and seas can be flat.

The power of mountains and seas is the decathlon. The ability to generate is very subjective, really when the scene is landed, language understanding ability is very important, why did I think it was an artificial intellectual disability before, because of the lack of understanding and code ability. The improvement of code capabilities can help improve the reasoning ability of large models, and the output results must comply with domestic laws and regulations and even moral values. We also adopt the architecture of GPT-4 plug-in to help enterprises and customers from data optimization, model training, model deployment and other one-stop services.

Why do large models have complex logical reasoning capabilities? We did it today, but I don't know why, whether 50 billion parameters or 100 billion parameters are better, but it is difficult to say, maybe the neurons in the 100 billion parameters have not been activated.

In addition, there is medical care, at the beginning we are making a large model, many people think that Yunzhisheng is doing vertical industry models, no, we are doing industry applications. Challenging one of the most serious scenarios - medical treatment, through the pre-training stage, we have collected a lot of medical literature, monographs, books, and medical cases, and accumulated tens of millions of real labeled data, which can be converted into our fine-tuning data.

In addition, in 2019, it also won the first prize of Beijing Science and Technology Progress Award, the winning project is the key technology and application of large-scale knowledge graph construction, we have one of the largest medical knowledge maps in China, we decompose the knowledge graph into knowledge plug-ins embedded in the large language model, so that the large model becomes an expert in the medical field.

MedQA is a very authoritative medical knowledge quiz test set, including Google's Med-PaLM, ChatGPT and GPT-4 have published their results on this test set, Shanhai recently achieved 81 points in the review, greatly exceeding GPT-4's 71 points. Through domain enhancement, it is possible to turn large models into experts in a certain field. There is also a number that can be compared horizontally, medical school graduates to pass the clinical practitioner examination, the current known AI maximum score is 456 points, Shanhai about 511 points, which is the super ability obtained by the large model through field enhancement.

It is still quite difficult to make a big model, the threshold is very high, in addition to the need for a lot of money, excellent algorithm engineers and algorithms, but also a lot of ability, we summarize it as the work of mountains and seas. Intuitively speaking, the big model itself is a big data set, the big model is the work of engineers, why can Yunzhisheng make a very authoritative objective evaluation data in a few months, we evaluate internally, not only in the medical treatment, in the general field, Yunzhisheng is one of the best.

The computing power platform is not how many cards to buy to plug in, Yunzhisheng almost has 200P computing power, the efficiency of the cluster has reached the top level in the industry, and we can train our model very quickly with relatively few cards.

Our current utilization rate of GPU clusters can achieve 50%, large models need multiple cards, and the current level in the industry is about 42%. Large models also need to achieve 3D hybrid parallel training. What is 3D? It is the parallelization of the model, the parallelization of the data, and the parallelization of the pipeline, which separates the task into different cards of many different machines to calculate separately, and finally can quickly get the response result. In addition, in the model reasoning has been a lot of optimization, the speed of reasoning has increased by 5 times, how to separate the training card and the inference card, the training card is A800, and the inference card can achieve fast reasoning on a single card A6000.

In addition, data is very important, data scale, data diversity, data quality, we can now support 10T level of fast deduplication, ChatGPT training number is 45T, but preferably after using hundreds of gigabytes of data to train.

With these capabilities, based on the capabilities of Atlas and UniDataOps, Shanhai's capabilities and industry customers can be better served.

Intelligent IoT is also an important business of the company, we have a lot of landing, the effect in the past is really not very good, I hope that after having Shanhai, use a large model to do all the existing Internet of Things products.

Medical care is the direction we are optimistic about. In the previous medical direction, the product mainly had two aspects, one was to directly talk with the microphone without typing on the keyboard by hand, which greatly improved the work efficiency of doctors and shortened the medical record input time from 3 hours to 1 hour; Second, after having medical records, there is also a system to review medical records through the AI brain, check whether there are errors in medical records, and now what can be done after having the ability of AI large models?

All conversations during the conversation are recorded and the key messages are identified as summaries of the information. With those key information that follows communication, you can guide the generation of cases at the touch of a button. In the past, doctors were required to read medical records word by word, but now they can form medical records based on key information.

Shanhai's vision is to create a connected, intuitive world through artificial intelligence, which was previously defined as making machines obey people, and today wants machines to be more anthropomorphic. The communication between people and things will become more intuitive, new capabilities will bring new products, new business models, and I am very willing to welcome the new era of big models with everyone here.