How to build intelligent applications in 10 minutes? ——The application development revolution brought by the big language model

Wedge: Siri for everyone

Long ago, humans came up with the idea of natural language interaction with computers, and the famous "Turing test" was also designed based on human-computer dialogue scenarios. In 1966, Joseph Weizenbaum developed the world's first chatbot, Eliza, at MIT. However, due to the limitations of technical conditions at that time, Eliza could only carry out limited pattern matching, and the dialogue ability was very limited, and the truly intelligent and natural dialogue ability has always been the "holy grail" in the field of AI. It was not until the 21st century that the development of statistical machine learning and neural networks made great progress in chatbots, but training a chatbot still requires massive amounts of data, computing power and research and development time.

When Apple first exposed its smart assistant Siri in October 2011, many thought Siri opened up "a whole new possibility for the future of mobile devices", but more than a decade later, Siri's expectations have been disappointed. Every user has encountered their own "moments of disappointment" on Siri — it may not understand a simple command, it may not be able to grasp even the most basic operation, and it is impossible to have some in-depth discussion with you, such as talking about AI technology schools, talking about European literary history, or simply making a rough travel plan. If this can be called "intelligence", then "intelligence" has been low to the dust.

Until the birth of ChatGPT at the end of 2022, the hope that people cooled in Siri in the past came alive again.

Build an intelligent chatbot quickly in 10 minutes

Today, with OpenAI's open API, you can quickly build an intelligent chatbot with common sense and rich knowledge in 10 minutes with only a few lines of code. The main steps are as follows:

1. Register an OpenAI account to obtain an API Key;
2. Install the OpenAI Python library;
3. Prepare the prompt and call Model to complete the conversation;

Here is the sample code:

import os
import openai

OPENAI_API_KEY = os.environ.get("OPENAI_API_KEY")
openai.api_key = OPENAI_API_KEY

def get_completion_from_messages(messages， model="gpt-3.5-turbo"， temperature=0):
    # 传递整个messages，可指定角色
    response = openai.ChatCompletion.create(
        model=model，
        messages=messages，
        temperature=temperature， # 控制模型输出的随机程度
    )
# print(str(response.choices[0].message))
    return response.choices[0].message["content"]           

When calling the API, in addition to selecting the model, you can also adjust the following parameters to generate different styles of text to achieve more suitable interactions:

• max_tokens: THE MAXIMUM NUMBER OF TOKENS GENERATED FOR TEXT, USUALLY THE NUMBER OF WORDS OR SUBWORDS;
• temperature: randomness parameters, lowering temperature will get more conservative and predictable results;
• top_p: nucleus sampling parameter, which controls the diversity of the generated text;
• n: Number of candidate responses generated;
• presence_penalty: the probability of repetition of penalizing existing content;
• frequency_penalty: the probability of penalizing the occurrence of high-frequency words;

The text generation interface provided by OpenAI is stateless, and the chatbot has no memory by default, but the solution is simple, that is, every time a question is asked, the historical conversation record is sent to the AI model. The system presets several different roles, the assistant is the chatbot itself, the system is the background setting of the robot, and the user is the user who asks the question. The screenshot above gives an example, the user tells the robot his name at the beginning, and then brings the historical chat record to the bot in the subsequent conversation, and the robot can use this context record to correctly "recall" the user's name.

Develop scenario-based intelligent applications around prompt words

It doesn't make much sense to just call the interface and develop a chatbot that is consistent with OpenAI's native capabilities, and more importantly, with the help of pre-trained large models, we can quickly develop intelligent applications suitable for specific scenarios. The key is to build a prompt so that the model is clear about the application scenario and expected results. Prompt can contain:

• Background information: provide the context of the application scenario for the model;
• Indication: Clearly tell the model what it should do;
• Examples: Examples that provide inputs and expected outputs;

For more on the principles and techniques of Prompt Engineering, see my other blog post: Questions Matter More Than Answers – 10 Minutes to ChatGPT Prompt Essentials.

In the above figure to give an example of intelligent customer service, you can see that a few lines of Python code are almost indistinguishable from the first example, but a large number of natural language descriptions are added to the program, one part of which is product information for query, and the other part (step 1 of the middle diagram...). Step 2, etc.) is a description of how to deal with the customer's problem.

delimiter = "# # # # "
system_message = f"""
请按照以下步骤回答客户的查询。客户的查询将以四个井号（# ）分隔，即 {delimiter}。
步骤 1:{delimiter} 首先确定用户是否正在询问有关特定产品或产品的问题。产品类别不算。
步骤 2:{delimiter} 如果用户询问特定产品，请确认产品是否在以下列表中。所有可用产品：
产品：TechPro超极本
类别：计算机和笔记本电脑
品牌：TechPro
型号：TP-UB100
保修期：1年
评分：4.5
特点：13.3英寸显示屏，8GB RAM，256GB SSD，Intel Core i5处理器
描述：一款适用于日常使用的时尚轻便的超极本。
价格：$799.99
……
步骤 3:{delimiter} 如果消息中包含上述列表中的产品，请列出用户在消息中做出的任何假设，例如笔记本电脑 X 比笔记本电脑 Y 大，或者笔记本电脑 Z 有 2 年保修期。
步骤 4:{delimiter} 如果用户做出了任何假设，请根据产品信息确定假设是否正确。
步骤 5:{delimiter} 如果用户有任何错误的假设，请先礼貌地纠正客户的错误假设（如果适用）。只提及或引用可用产品列表中的产品，因为这是商店销售的唯一五款产品。以友好的口吻回答客户。
使用以下格式回答问题：
步骤 1:{delimiter} <步骤 1的推理>
步骤 2:{delimiter} <步骤 2 的推理>
步骤 3:{delimiter} <步骤 3 的推理>
步骤 4:{delimiter} <步骤 4 的推理>
回复客户:{delimiter} <回复客户的内容>
请确保在每个步骤之间使用 {delimiter} 进行分隔。"""           

This development mode of describing business logic in natural language is the biggest difference between application development based on large language models and traditional intelligent application development, which greatly reduces the development threshold, and ordinary users can participate in AI application creation. Traditional machine learning methods require collecting large amounts of conversational corpus and then training language models, and the process of building and tuning takes at least several months. The way Prompt is written is very free and flexible, by embedding role settings in prompt, you can let the chatbot simulate different personalities, by presetting keywords in advance, you can achieve scenario-based question and answer services, no need to train the model independently, or even no need to tune, write a few paragraphs of text description can make the chatbot adapt to different scenes, and achieve good results, such an instant experience is unimaginable before.

Furthermore, if you want to develop complex production environment applications, there are still many inconveniences in directly calling OpenAI APIs, such as the actual business scenarios often require multi-step processing, then we can use open source frameworks such as LangChain. It provides developers with the tools they need to create applications driven by LLM (Large Language Model), such as for multi-step processing, LangChain provides functional components called "chains" that allow developers to "chain" different components together to create more advanced LLM applications.

LangChain's core concepts include:

LLM Large Language Model: It is essentially a wrapper for a large language model, encapsulating differentiated interfaces for different large language models.

Chains: Provides the ability to invoke multiple processing results and chain them together to achieve complex goals. For example, retrieving data from a specific URL, summarizing the retrieved text, and using the resulting summary to answer questions.

Prompts Tips: Simplifies the creation and management of Prompts with specialized classes and functions, including prompt templates.

Document Loaders and Utils Document Tools: Document loaders convert various data sources to text for processing, while the utils module provides a variety of processing tools and code snippets.

Vectorstores vector storage: Use an embedding model to vectorize and store each document in a database for subsequent efficient retrieval of related documents through the vector.

Agents: Agents can invoke tools to complete flexible, indeterminate complex tasks based on user input.

The large model is based on the local knowledge base of Q&A, as a typical scenario use case, you can refer to the previous blog post: Create an AI personal assistant that really understands you! How to provide a local knowledge base for ChatGPT?

The application development revolution brought about by the big language model

Compared with the knowledge engineering methods of the past, the pre-trained large language model solves the knowledge acquisition bottleneck commonly faced by traditional AI systems. It is like an omniscient Baixiaosheng, with extensive knowledge and strong language understanding ability, which allows us to directly complete business programming through natural language description, without the need for manual large-scale data preparation, cleaning, annotation, model tuning, algorithm writing and other heavy labor.

Compared with the traditional machine learning method of intelligent application development, application development based on large language models has the following differences:

• Training data: No need to manually label data, large models can complete transfer learning through Prompt prompts;
• Development mode: based on natural language programming, no longer need to write rules, feature engineering, etc.;
• Development experience: interactive, intuitive, human-like language, no need to understand machine learning principles;
• Development efficiency: a few lines of code to prototype and greatly reduce development costs;
• Scalability: Optimize and expand model capabilities by continuously providing new samples without having to retrain from scratch;

Of course, large models also face problems such as huge computing power consumption, high training costs, and systematic bias. We still need to remain sober and cautious in applying this powerful new tool for the benefit of humanity. However, these changes have undoubtedly liberated the productivity of AI application developers, allowing them to focus on higher-level user needs, rather than repetitive and boring work such as feature extraction and model training parameter tuning. At the same time, natural language replaces traditional programming languages, allowing non-technical people to participate in AI creation, which will bring revolutionary changes.

Perhaps, we are standing on the threshold of a new era of artificial intelligence, and super-large-scale language models are like red-hot steel in a furnace, emitting a blazing light. It illuminates the last mile from the human brain to machine applications, and more intelligent applications based on large models are bound to develop rapidly, and the whole world will be different because of it.