Today's sharing is the artificial intelligence AI industry research report: "2023 Artificial Intelligence Big Model Evaluation Report". (Report produced by: Xinhua News Agency Research Institute)
A summary of the study is provided below
In 2023, major manufacturers have laid out large models, and their intrinsic value is undoubtedly revealed
The development of large models can provide more intelligent application scenarios, expand more intelligent services, improve service quality and efficiency, and realize industrial intelligent transformation. At the same time, the development of large models can provide more advanced intelligent products, improve the intelligence level of products, realize product upgrading, and enhance product competitiveness. In addition, the development of large models can also realize full-link commercial services from technology to products, improve the quality of commercial services, and expand more commercial service scenarios.
Overview of large model manufacturers in 2023
At present, typical domestic large models include: Alibaba's M6, Baidu's Wenxin model, Huawei's Pangu, ChatGLM of Zhipu Technology, iFLYTEK's Xinghuo, SenseTime's Ridayxin, etc., and other companies will also compete to enter the game in 2023 to jointly create a complete ecological link of China's large model.
Rules of this evaluation:
The versatility and generalization of large models have attracted the layout of major manufacturers, but there are very few evaluations of the actual effect of large model products on the market, mainly because the performance of large models is affected by a variety of factors, such as model architecture, training data, training algorithms, etc., and the evaluation system is too complex. In order to promote the development and application of large model technology, the China Enterprise Development Research Center of Xinhua News Agency Research Institute evaluates large model products through multiple dimensions (4 categories, 36 sub-capabilities, a total of 300 questions), and invites an expert team to deeply analyze the rationality and semantic expression of each product's answers, and finally obtains the evaluation of each manufacturer, in order to provide a directional reference for the future development of enterprises and the adoption of large models in the industry.
The evaluation latitude of this large model:
• Basic Capabilities (100 questions): Examine the product's language ability, cross-modal ability, and AI guidance ability for good.
• IQ test (100 questions): covers three major items: general knowledge knowledge, professional knowledge, and logical ability. Among them, professional knowledge includes 10+ subdivisions such as mathematics, physics, finance, literature, etc., and logical ability includes 6 dimensions such as reasoning ability, induction ability and summary.
• Emotional intelligence test (50 questions): measures the emotional ability of individual products. Including self-awareness, self-regulation, social awareness, interpersonal relationship management, etc., this emotional intelligence test focuses on emergencies, communication skills, emotional management, etc. in different scenarios.
• Work efficiency improvement ability (50 questions in total): For five types of workers, journalists, painters and designers, marketers, lawyers and researchers, the problems encountered by staff will be sorted out one by one, and whether the product can effectively help the work efficiency of relevant personnel.
Excerpts from the original text of the report are as follows:
(This article is for informational purposes only and does not represent any investment advice from us.) For information, please refer to the original report. ）
This report totals: 62 pages. Due to space limitations, only some of the contents are listed.
Featured reports from the public: [Phantom Vision], reply to the keyword "202306" to get the full PDF electronic version