The emergence of ChatGPT has triggered a new round of generative large model boom, and in the past six months, many companies at home and abroad have launched a "100-model war" and competed to launch their own large models. While technology enterprises are "involving", how to realize the landing of large models in the industry has become one of the most concerned issues from the outside world.
According to the "Research Report on the Map of Chinese Intelligent Large Model", China has entered a period of rapid development of large models since 2020, and a number of general-purpose large models are developing rapidly. According to incomplete statistics, up to now, 79 large models above the scale of 1 billion parameters in China have been released, and related application industries are rapidly expanding from office, life, entertainment and other directions to medical, industrial, education and other fields.
In the medical field, how to use large models to provide doctors with new tools and methods for disease diagnosis and treatment to improve disease diagnosis, treatment and prevention is also an urgent topic of concern in the industry.
Clinical problems are complex and difficult to cover? Multimodal medical basic model group to "break the game"
At the SenseTime "Boundless Love, New Day" Artificial Intelligence Forum, Dimitris N. Metaxas, Distinguished Professor of Computer Science at Rutgers University, pointed out that with the advent of the wave of large models, medical base models have shown good ability in solving medical multimodal data analysis and clinical long tail problems, such as in some clinical scenarios with less data, large models can bring significant AI effect improvement. In the future, generative and multimodal large models will continue to bring innovation to the healthcare industry.
Professor Dimitris N. Metaxas shared his views on large model empowerment at the forum
However, compared with other fields, the implementation of large models in the medical field faces more complex challenges. On the one hand, between medical data of different modalities, imaging modes and image features are quite different, and the long tail problem of medical clinical is complicated. On the other hand, the privacy requirements of medical data are extremely high, and meeting the deployment requirements of the local environment of medical institutions is also an important consideration for the implementation of large models. Therefore, the development of the industry urgently needs a large medical professional model that can adapt to different clinical problems and meet the requirements of clinical deployment.
It is understood that SenseTime cooperates with industry partners to launch a variety of vertical basic model groups such as medical language big model, medical image big model, and bioinformation big model, covering different medical data modes such as CT, MRI, ultrasound, endoscopy, pathology, medical text, and bioinformation data.
SenseCore, a large device with 5,000 Petaflops, supports simultaneous training of 20 ultra-large models with 100 billion parameters, greatly accelerating the innovation and application efficiency of large models. With the massive computing power of SenseTime's devices and the solid foundation of the medical basic model group, SenseTime can become a "medical model factory", based on the specific needs of medical institutions, help them efficiently train models for downstream clinical long-tail problems, and even assist institutions to achieve model independent training.
"This model breaks through the bottleneck of small data samples and high labeling difficulty for medical long-tail problems, realizes small data, weak labeling, and efficient training for different tasks, and significantly reduces the deployment cost of large models to meet the personalized and diversified clinical diagnosis and treatment needs of different medical institutions." Zhang Shaoting, vice president and head of smart medical business of SenseTime, said that this model takes into account the balance between large model performance and landing convenience, helping medical institutions to reduce costs and increase efficiency while solving difficult diseases.
With the help of large computing power and a solid foundation of large models, SenseTime empowers clinical diagnosis and treatment in an all-round way
At present, SenseTime has integrated the medical basic model group into the "SenseCare® Smart Hospital" comprehensive solution to better promote the high-quality development of hospitals.
Taking advantage of the depth, SenseTime's model realizes the landing of multiple medical scenarios
Industrial application scenarios have become the best "training ground" for large models, and vertical large models will further exert their in-depth advantages and promote the rapid development of the industry. SenseTime actively explored the implementation of large models in the medical field, and during WAIC, released an upgraded version of the "SenseCare® Smart Hospital" integrated solution integrated into the large medical model, covering five scenarios: smart medical treatment, smart diagnosis and treatment, smart hospital management, smart medical research, and smart medical cloud, creating a five-in-one all-round product matrix. At present, the solution has been landed in many top tertiary hospitals across the country, using more diversified AI services to empower hospital diagnosis and treatment, scientific research, decision-making and other levels, and help hospitals transform and upgrade intelligently.
The First Affiliated Hospital of Zhengzhou University undertook the construction of the National Telemedicine Center, creating a leading and internationally advanced telemedicine service system with seven levels of linkage of "international-national-provincial-city-county-township-village". SenseTime used the big medical model and solid smart medical cloud platform construction capabilities to help the First Affiliated Hospital of Zhengzhou University build an integrated multi-modal integrated service platform and innovate the global telemedicine health service model in Henan Province.
At the specific application level, with the empowerment of the medical big model, the First Affiliated Hospital of Zhengzhou University has built three intelligent remote diagnosis platforms of imaging, pathology and ECG, responding to the needs of grassroots hospitals, and assisting the diagnosis and treatment of regional characteristic diseases and difficult diseases with the help of large models, effectively promoting the solution of medical long-tail problems. In addition, based on massive pharmaceutical knowledge and expert experience, the hospital has cooperated with SenseTime to develop a cutting-edge drug consultation language model in the industry, which will provide patients with a smarter, more convenient and efficient online medication consultation service and experience. Zhao Jie, deputy secretary of the Party Committee and vice president of the First Affiliated Hospital of Zhengzhou University, pointed out that with the development of various new technologies such as artificial intelligence, telemedicine will be comprehensively improved in terms of service content, service methods, service applications and service management in the future.
SenseTime assisted Zhengda First Affiliated Hospital to build a multimodal intelligent remote diagnosis platform for imaging, pathology, and ECG, enabling the improvement of primary medical care
SenseTime assisted Ruijin Hospital to build a hospital-wide smart image cloud platform, covering various functions including image data interconnection, mobile reading, 5G+ remote consultation, AI intelligent diagnosis and treatment, and patient cloud image health records.
In terms of precision diagnosis and treatment, with the assistance of the multimodal medical basic model, SenseTime has helped Ruijin Hospital realize AI-assisted diagnosis and treatment of multiple parts and organs of the whole body, providing intelligent services for multiple campuses and clinical departments of Ruijin Hospital that meet the whole process of diagnosis, treatment and healing, effectively improving the efficiency and accuracy of doctors' diagnosis and treatment. Next, SenseTime will explore the implementation of the medical model in more medical scenarios, empower more departments, optimize the diagnosis and treatment effect of doctors, and improve the medical experience of patients.
SenseTime helped Ruijin Hospital to build a hospital-wide smart imaging cloud platform at WAIC
For the medical treatment link, the smart medical platform built by Negotiation Technology based on the medical language model "Big Doctor" can provide various functions such as intelligent guidance, intelligent pre-consultation, pre-diagnosis inspection, intelligent follow-up, and in-hospital navigation, realizing the digital upgrade of multiple links before, during and after diagnosis, reshaping the patient treatment process, realizing the intelligence of the whole process of the outpatient link, shortening the patient's stay time in the hospital, and optimizing the patient's medical experience.
"Intelligent Companion Assistant" can provide intelligent guidance, in-hospital navigation and other functions
Relevant people from SenseTime said that in the future, the company will further focus on the demand scenarios of different medical institutions, further expand the "medical model factory" based on the medical basic model group, and provide customers with one-stop clinical solutions with more clinical functions and more innovative service forms such as MaaS (Model-as-a-Service), so as to better help the intelligent transformation and high-quality development of hospitals and the realization of inclusive healthcare.