The landing of large language models Why the first step is to do customer service

After almost two years of blowing on the cusp, when practitioners talk about artificial intelligence large language models again in the spring of 2024, "landing" has become the most talked about word.

"Enterprises hope that the large model can achieve group-level empowerment, but it is difficult to find an entry point, and it is recommended to take intelligent customer service as the first stop on the road to the implementation of the large model." At the recently concluded Zhongguancun Forum, when the Beijing Municipal Science and Technology Commission released the "Beijing Artificial Intelligence Large Model Industry Application Analysis Report", this sentence was specially marked on the big screen.

Why has customer service become the first stop for the landing of large models? In this regard, the Beijing News Shell Finance reporter interviewed the developers of the large model and the enterprises with landing needs, and found that the large model can indeed bring a lot of ability to understand user questions, answer user questions continuously and 24 hours a day, etc., although there is still a gap from the real person, but counting the cost account, the large model has been "usable".

However, there are also consumers who "complain" to reporters, such as in the online shopping scene, although AI customer service is more "silky" than in the past in dialogue and appears more emotionally intelligent, there will also be situations where the answer is not asked. In the view of many practitioners, although AI customer service can reduce costs, if you want to "go further", change from usable to easy to use, and shine in more scenarios, large models need to be further upgraded.

On May 9, in a large-scale model application construction training camp, enterprise developers asked the organizer questions about the technical problems encountered in the implementation of AI

24×7 working AI on-the-job customer service

"We started using large-scale model technology for customer service in June last year." On May 9, an employee of a Beijing start-up company told a reporter from Shell Finance, "The large model technology mainly solves the problem of continuous questions from customers, if you use human customer service, customers have to repeat the problem every time they ask a question, but the large model can remember the customer's previous questions, and it is more logical than human customer service." In addition, it can work 7X24 hours, and manual customer service cannot. ”

In fact, AI has been widely used in customer service long before the advent of large language model technology. "Originally, we did AI by defining a bunch of tasks, but in many places in the sales and service process, we couldn't think of what kind of way customers would ask and how to impress customers. After the large model came out, it happened that the problem of scene generalization could be solved through the ability to understand and reason about language, including the ability to understand and reason with preliminary AGI (general artificial intelligence). Kong Miao, deputy general manager of Ronglian Cloud Industry Digital Cloud Business Group, said in an interview with a reporter from the Beijing News Shell Finance.

In his opinion, compared with the idea that "large models can solve everything", the AI landing value is not worth doing, and we should focus on the production ratio, "The original intelligence needs continuous investment, so the production ratio is extremely low, and the large model through external data and the ability to identify and generalize intentions makes us now able to solve the problem that intelligent dialogue cannot be carried out in the past and must be recalled, when 90% of the recall is reduced to 10% of the recall, it also improves the available scenarios." ”

From the demand side, enterprises in the industry can intuitively experience the convenient services brought by large models, enhance the group identity within the enterprise, and reduce internal resistance for the subsequent application of large models in the whole industry chain. From the technical side, the intelligent customer service application scenario is one of the general scenarios of the large model, which is relatively less technically difficult, and can give full play to the technical advantages of the large model, which is conducive to the rapid implementation of the large model.

In the interview, the Beijing News Shell Finance reporter found that there are many ways for the company to introduce AI large model capabilities to make AI "land", including technical personnel directly downloading open source large models, cooperating with large model companies to privatize large model applications, and using a one-stop large model service platform. Among the above three methods, downloading the open source model is free, while the last two are the software sales model and the free use model but charging computing power fees by token (statement). The above-mentioned Beijing start-up company employee told reporters that the latter two ways can actually be chosen independently, "If you have a limited budget, you can choose a smaller model, a cheaper model." In addition, when using the one-stop large model service platform, there are two types of fees: time-based buyout fees and time-based computing power consumption, so you can choose the most money-saving method according to your busy and idle hours. ”

"In the era of non-large models, many questions and answers are customized, and the answers are relatively blunt. With the improvement of model capabilities, more and more industries are actively embracing large models, such as mobile phones and autonomous driving. As technological capabilities change, so do new business needs, which are a series of breakthroughs and innovations. On May 9, Zhou Jingren, CTO of Alibaba Cloud, said in an interview with a reporter from the Beijing News Shell Finance.

Large models are not perfect "substitute for human" or can be changed into "auxiliary labor"

However, will AI models completely replace human customer service? At least for now, the answer is no.

Talking about the online shopping experience of buying a skirt at the end of March, Ms. Liu from Tianjin realized after a long time that the customer service she had chatted with for 10 days was actually AI, "At the beginning, I asked some questions about the product, such as whether the skirt was elastic and had a zipper, and the other party could answer it clearly." But since it hasn't been delivered, I started to negotiate with the other party, and when I asked directly, 'When will this blue model be shipped?', the other party began to repeatedly say that it would be sent as soon as possible, but it just couldn't tell me the exact delivery time, so I complained about the store in a fit of anger. ”

Chen Yinjiang, deputy secretary-general of the Consumer Rights Protection Law Research Association of the China Law Society, said that for enterprises, the promotion of intelligent customer service is more often to reduce costs. Internet transactions have the characteristics of massive and instantaneous transactions, and sometimes it is necessary to invest a lot of customer service forces, and the appropriate promotion of intelligent customer service can greatly improve service efficiency. However, many intelligent customer service programs are difficult to meet the actual needs of consumers, so the key is to retain manual customer service at the same time, so that consumers can choose which customer service method to choose.

In addition, if you give AI customer service too high permissions, it may also cause risks to the enterprise. In December 2023, a foreign user induced an AI customer service of a car dealer connected to the ChatGPT interface to sell a Chevrolet for $1, and even after the car dealer blocked the relevant loopholes, another user pretended to be Sam Ultraman, the founder of OpenAI, and induced the car dealer's robot to give away the vehicle for free, resulting in the car dealer having to turn off the AI customer service function, which fully shows that the current AI chatbot is far from the real judgment ability of human customer service.

In fact, AI customer service and human customer service may not simply replace the relationship. "In the past, we had intelligent sales assistance, but more often than only through SOP (standard operating procedures) to solve the problem of getting started with the front-line language of the enterprise, the large model can be generalized capabilities, more experience can be copied to the team, the more the sales ability at the bottom will be improved, so as to achieve the efficiency of the entire business." Kong Miao said.

What problems need to be solved in the implementation of AI?

At present, the large model technology is still in continuous development, in this process, how to make the large model land and available according to the existing capabilities, the reporter interviewed a number of employees of the enterprise who have tried the large model on the B side.

Specifically, if a company wants to implement AI in its own scenarios, the first step is to first establish the company's own knowledge base, then select the existing large model, and "plug in" the company's knowledge data into the large model in order to generate usable large model landing applications. "In the past, we relied on QA (Q&A) to do knowledge maintenance, and we found that a large amount of knowledge could not be disassembled through manual or traditional rules, but we can manage multiple documents through the copilot (copilot, referring to the large model intelligent assistant) knowledge base, and then combine the large model with RAG's technology to form a template, and now we can also solve the problem of knowledge flow in the entire marketing service." Kong Miao said.

However, in the process of landing the large model, there are various problems encountered in different industries and different scenarios.

Some employees of the company's technical department engaged in video training business reported to the Shell financial reporter that their company's data includes text, pictures, and videos, among which the way to import video knowledge into the knowledge base is mainly through "tiling", but this result is not very accurate. Some people engaged in cross-border financial statistics said that the database software in the financial field and the data table format and standards of overseas users are not uniform, so there is a problem of sentence conversion when importing large models. In addition, there is also a common problem that the company itself does not have enough data reserves, so it is worried about the effectiveness of the large model to answer the problem.

And after the big model is built, the same problem exists. Some practitioners in the automotive industry who use large model cloud services said that they hope that the large model can analyze the emotional characteristics in customer feedback messages in work orders, but they do not want to leak customer information. The employees of the above-mentioned video training company said that for some relatively difficult questions, the large model cannot guarantee to give 100% accurate responses, and this "illusion" problem cannot be solved.

"Since the launch of the large-scale model technology, it has brought us a lot of imagination, including communication and dialogue like a real person, but after high expectations, when it is really implemented in the industry and customers, we still find many limitations, such as computing power bottlenecks and illusion problems." Kong Miao admitted to the Shell Financial Reporter, "Therefore, in the actual application of the large model in the enterprise landing scenario, many non-standard things need to do some reasoning, and some technologies need to be supplemented, such as the introduction of search methods for mathematical problems, and the transformation of the large model from a liberal arts student to a 'liberal arts and sciences', which are all product engineering work." ”

Zhou Jingren told reporters that the time has come for the large model to land, if the basic model itself is not strong enough, there will be challenges in the landing process, but in the past year, the ability of the large model has been greatly improved, and many previously impossible scenarios have now become possible.

In the interview, a number of large model developers told reporters that the current is far from the limit of large models, and over time, new technologies will be expected to solve the current problems, such as technical personnel for example, in the process of multiple rounds of dialogue, for financial analysis and customer service scenarios, compared with the RAG method, the use of long text to solve the problem is more appropriate. In addition, product engineering work is also indispensable, such as the problem that video knowledge content is difficult to import into the knowledge base, some technicians said that the problem can also be solved by selecting three small models for subtitles, slices, and languages, "Of course, I believe that maybe in a few months, a single large model can solve this problem through technical upgrades." ”

Reporter contact email: [email protected]

Beijing News Shell Financial Reporter Luo Yidan Editor Wang Jinyu Proofreader Lu Qian