With the change of international trends, intelligent manufacturing has become a new direction for the mainland manufacturing industry, and a large number of outstanding enterprises have entered a critical period of transformation and upgrading and enhancing competitiveness with years of technological advantages. Knowledge inheritance and innovation are the driving force for enterprise development, and effective knowledge management and application are the key factors in improving R&D and manufacturing experience, optimizing business efficiency and quality, and shortening production cycles in the automotive field.
First, the current problems in knowledge management
(1) The knowledge management of the automotive industry is relatively scattered, there is a lack of unified management system and standard specifications, data retrieval is difficult, coupled with the relative independence of each department, knowledge sharing is superficial, resulting in knowledge data is not easy to be reused and referenced.
(2) the demand for knowledge management throughout the automotive industry is increasingly diverse and specialized, and it is not only expected to be able to deeply customize it according to its own field, so as to improve the accuracy and reliability of the model; Enterprises are also highly concerned about data security issues, hoping to ensure data security to the greatest extent through privatization and deployment.
(3) Some small and medium-sized enterprises hope to use knowledge graphs and large language models to build their own knowledge bases, and because the technical threshold is also high, enterprises also expect knowledge graph and large language model providers to provide a variety of interfaces and protocols to facilitate seamless integration and calling.
(4) Considering the large investment in data storage and computing power in massive knowledge management and mining, even benchmark enterprises in many industries are seeking a balance point in cost.
In this context, with the competitive trend, strategic development requirements and business improvement needs, the use of artificial intelligence technology to help automotive enterprises carry out the construction of knowledge management graph application has become more and more urgent, and localized, customized and industrialized knowledge graph and large language model have also become an important development direction.
Second, the background of the birth of KG-LLM
(1) The advantages and disadvantages of knowledge graph and large language model
The knowledge graph cares more about rules, while the big language model is a universal undifferentiated service.
Knowledge graph is a structured and standardized way of knowledge representation, which is to analyze, integrate and mine various information, transform this information into a calculable knowledge system, and display it in the form of a map. For example, there is a definition of P (that is, relationship) displayed in the map, but in the language, when speaking, you may not directly say a certain relationship, or say some aliases of that relationship. In language, relationships are mostly implicit, which is inherently at odds with explicit structures in graphs. In addition, relational aggregation in the graph can be a challenge.
Large language models are more suitable for natural language understanding and generation, and the resulting knowledge structure is based on statistical analysis of a large amount of text, and the model can be trained to predict the next possible word or phrase. Big language models exist more to solve problems in natural language processing than to build knowledge graphs.
Knowledge graphs and large language models can also be combined, for example, combining knowledge graphs with pre-trained language representation model BERT can enable machines to use relevant domain knowledge to reason when reading domain-specific text, thereby improving the performance of the model.
When using a knowledge graph to enhance a pretrained model, we expect the pretrained model to perform better on tasks that require specific knowledge or common sense. For example, in machine reading comprehension, given a sentence from a Harry Potter novel: "Harry Potter points his wand at Lord Voldmort.", if the model is to understand this sentence, such as the space-time relationship between Harry Potter and Voldemort, the model needs to know some prior knowledge of Harry Potter, so that the model has a better understanding of certain entities in the text, not just as a title (referent).
(2) What a knowledge graph can do for a large language model
Large language models can use knowledge graphs for knowledge augmentation to improve their reasoning ability. However, explicit knowledge representations in graphs differ significantly from representations in languages, such as the definition of relationships shown in graphs, but in languages some relationships may be expressed implicitly or by aliases. In addition, the aggregation of relationships in the map is also a difficult problem, such as A->P1->B and B->P2->C, what should be the relationship between A->C, that is, what should be P1+P2, which is a problem that cannot be covered by explicitly introducing complex subtrees. Therefore, it is not feasible to conceal P in the knowledge enhancement process, because it will make it difficult to control the scale of pulling additional knowledge from the knowledge graph, because the relationships in the knowledge graph are not necessarily transitive.
On the other hand, code pretraining can enhance the reasoning ability of large language models, especially when mixing code with natural language pretraining. In this case, the reasoning ability of the model has been greatly improved, even if no specific inference method is adopted, just switching from a plain text pre-training model to a text and code hybrid pre-training model, the model reasoning ability has been greatly improved on almost all test data sets. This may be because the code training data contains a significant proportion of the code, descriptions, and comments for mathematical or logical problems that are helpful for solving downstream mathematical reasoning problems.
Therefore, large language models can use knowledge graphs for knowledge enhancement, but need to solve the problems of explicit knowledge representation in graphs and expressions in language and relationship aggregation. In addition, code pretraining can also improve the reasoning ability of large language models, especially when mixing pre-trained code and natural language. These findings provide new ideas and directions for improving the reasoning ability of large language models.
(3) What a large language model can do for a knowledge graph
Knowledge graph is through the analysis, integration and mining of various information, the information is transformed into a calculable knowledge system, and displayed in the form of a map. This representation of knowledge is more structured, normalized, and better readable and understandable. The big language model is more suitable for the understanding and generation of natural language, but the knowledge structure it produces.
Large language models can be used in conjunction with knowledge graphs to improve the accuracy and richness of knowledge, and enhance the model's understanding and application of domain knowledge. In the process of establishing the knowledge graph, the big language model can be used to improve the efficiency and accuracy of knowledge extraction and knowledge injection. Here are some ways to take advantage of large language models:
Models such as ERNIE and KnowBERT that directly use the map representation vector as feature input. These models input entities and relationships from the knowledge graph as features into the language model to help the model better understand the entities and relationships in the text, thereby improving the accuracy of knowledge extraction.
Models such as KEPLER and WKLM for knowledge injection are realized by designing new pre-training tasks. These models inject entities and relationships from the knowledge graph into the model by designing specific tasks during the pre-training process, such as tasks such as relationship prediction and knowledge completion, to help the model better understand the entities and relationships in the text.
BY ADDING ADDITIONAL MODULES TO THE K-ADAPTER AND OTHER MODELS. These models inject entities and relationships from the knowledge graph into the model by adding additional modules, such as the knowledge injection module and the relationship inference module, to help the model better understand the entities and relationships in the text.
In the process of establishing a knowledge graph, it is necessary to carry out steps such as data collection, knowledge definition, knowledge storage, knowledge extraction, knowledge fusion, knowledge computing and knowledge application, so as to finally build a complete knowledge graph.
Third, the industry opportunities brought by KG-LLM
Automotive enterprises should actively respond to technological development and business needs, rely on the whole business chain of automobile design and development, manufacturing, after-sales operation and maintenance, etc., and adopt self-developed knowledge graph-based large language model (KG-LLM) technology to carry out the construction of their own knowledge management technology support platform.
By using the large language model in the process of knowledge graph construction to improve the efficiency and accuracy of knowledge extraction and knowledge injection, and combining the knowledge graph with the pre-trained language representation model BERT, the machine can use relevant domain knowledge to reason when reading the text in the automotive field, so as to improve the performance of the model, better enhance the model's understanding and application ability of domain knowledge, and effectively respond to various challenges of enterprise knowledge management and application.
The Big Language Model (KG-LLM) application platform based on automotive knowledge graph can comprehensively aggregate, refine and integrate internal and external knowledge, realize the unification and standardization of internal and external explicit or tacit knowledge, continuous sharing and inheritance, and support the mutual collaboration of knowledge management and application in the whole chain cycle of automobile manufacturing, with the following advantages:
(1) KG-LLM focuses on the application of vertical fields, which can provide more comprehensive and accurate knowledge management and application capabilities for automotive vertical fields, break through the "barriers" of knowledge, and enable knowledge to be freely shared and disseminated in the whole chain; It can also effectively link past design models with expert analysis, and realize continuous optimization and updating of knowledge through practical experience feedback.
(2) The training data and algorithm models required by enterprises applied to KG-LLM can be independently controlled, which helps to meet localization needs and reduce dependence on external resources, and is expected to ensure data security and model autonomy while meeting the flexible access and localization of enterprises.
(3) KG-LLM can be privately deployed for enterprises, so as to effectively ensure data security and privacy.
4. Typical application scenarios
The big language model (KG-LLM) based on knowledge graph can be applied to the whole business chain of automobile design and development, manufacturing, after-sales operation and maintenance, etc., which can improve production efficiency, reduce failure rate, improve product quality, and help enterprises better understand and meet the knowledge application needs of users.
(1) Design and development
KG-LLM can assist automotive R&D teams to understand and integrate complex design requirements during the design phase, help design teams better define new product specifications by parsing design goals and constraints proposed in natural language, and provide recommendations based on existing research and design.
(2) Manufacturing
KG-LLM can help analyze and optimize the production process, can find out the bottlenecks and inefficiencies in the production line, and then give suggestions to improve production efficiency, and can also help enterprises better manage inventory and adjust production plans through the analysis of various production indicators.
(3) Fault diagnosis and prediction
KG-LLM can help after-sales personnel analyze fault reports, give possible fault causes and repair solutions, and learn from historical fault data to predict possible failures, so as to take preventive actions in advance.
(4) Technical document management
KG-LLM can analyze a large number of technical documents and reports generated in the automotive development and production process, extract important information for technicians to quickly understand and use, and can automatically generate document summaries, or convert long technical reports into easy-to-understand language.
(5) Quality control and feedback
KG-LLM can analyze production data and quality reports, identify quality problems and provide improvement suggestions, and can also analyze user feedback to help enterprises understand user needs and expectations, so as to better meet user needs in future product design.
V. Summary
It is foreseeable that the automotive field will gather the process data information of each link around the whole process of automobile production, and after using the innovative knowledge graph-based large language model (KG-LLM) as the engine to participate in enterprise management, it can effectively solve the core problems of decentralized management of resources, time-consuming and laborious search, inability to obtain comprehensive relevant knowledge, and empirical knowledge that cannot be effectively inherited and applied, better optimize knowledge management in the automotive field, and lay the foundation for subsequent promotion and application. So as to provide new impetus for the transformation and development of enterprises!