On November 6, Western time, OpenAI held its first developer conference. In just 45 minutes, OpenAI has released a number of new products and features, including a new model GPT4-Turbo, user-customizable GPT, and a new Agent framework, Assistant API, etc. Here, we analyze what impact it has brought to the AI ecosystem, and how these impacts will reshape the future entrepreneurial landscape.
01 DevDay at a glance
In this DevDay, OpenAI's blockbuster features/products mainly include:
New model GPT4-Turbo
- Longer Context: Upgrade to 128K, which is equivalent to a 300-page book.
- Cheaper: Compared to GPT4, the overall price of GPT-4 Turbo is reduced by 2.75 times overall.
- More accurate: Developers can have more control over the model, introduce JSON, call multiple functions, and introduce reproducible output to make the model output more stable and accurate; The knowledge base is updated as of April 2023.
- Multimodal API: Multimodal Vision, DALL· E 3 is open together with the Speech Synthesis API.
- In addition, OpenAI has also opened up fine-tuning of GPT4.
GPT Builder & GPT Store - Build an application ecosystem around ChatGPT
- Users can customize ChatGPT through natural language interactions without writing a single line of code. User-defined GPTs can be combined with the user's own data and information with the basic model, and at the same time connected to APIs to perform tasks, such as writing emails, sending and receiving text messages, and managing databases.
- GPT Store will be launched later, forming an application ecosystem around ChatGPT. OpenAI will split the revenue with developers.
Assistant API - an in-app agent for developers
The Assistant API helps developers build agents in their own applications, which is equivalent to the official agent development framework based on large models, including: persistent and infinitely long threads, Retrieval, Code interpretors, and Function Calling.
02 Analyze DevDay
As the most high-profile leader in the field of AI, OpenAI's every move affects the future direction of the industry. Share our observations on this DevDay.
At present, OpenAI's strategy is very clear, it is a two-legged approach: one is to improve the basic model capability (pointing to AGI), and the other is to build a developer ecosystem and application platform. The focus of this "developer" conference is on the latter, and it is expected that next year's GPT5 conference may focus more on the former. The new products/features released this time, such as Longer Context, JSON Mode, Customizable GPT, Assistant API, and Multimodal API, all enable developers to develop applications more easily, flexibly, and efficiently. The reason why OpenAI cares about the construction of the application ecosystem is that it hopes to occupy as many user scenarios as possible and expand the user base in the early stage. At present, the most important resources for the top players of large models are computing power and data, and building a developer ecosystem, allowing more developers to create better products, and allowing users to continue to pour into the ecosystem is the foundation.
OpenAI has built ChatGPT's application ecosystem through GPT Builder and GPT Store. This is actually a revamp and upgrade of the ChatGPT Plugin Platform released at the beginning of the year. The intent of the plugin platform is to make ChatGPT a super traffic portal, where users can complete various tasks within ChatGPT through the plugin, and ChatGPT will become a large traffic distribution platform. However, the plugin has encountered various problems in the implementation and use, and it has not really become popular. This time, GPT Builder and the upcoming GPT Store are still aimed at traffic distribution, but the format has been changed: on the user side, it is presented in the form of "application distribution" (instead of letting users choose relevant plug-ins), and the user experience is better. From a developer's point of view, it also provides greater flexibility. Developers can integrate more relevant information, API calls, etc. in a Custom GPT, which is more likely to completely solve the user's problems in a certain scenario than the plugin. At the same time, its mechanism for the distribution of benefits is also more clear. It remains to be seen whether GPT Store can become an industry-changer like the App Store, but this has brought a relatively big threat to those projects that were originally "GPT shells", such as projects that do Q&A customer service based on 2B and 2C knowledge bases, and now users can easily build similar applications through GPT Builder.
The release of the Assistant API actually provides developers with an LLM-based agent development framework, making development more flexible and efficient. This has also greatly squeezed the living space of many LLM-based open source application development frameworks. The long-term value of a company lies in the fact that it either has a large model or an application (user/scenario/data). A company that has no control over either will have a sharp decrease in its survival space and value in the future.
03 DevDay's impact and reshaping of the entrepreneurial landscape
Computing power is the core factor affecting the evolution speed of large models, and whether an ecosystem can be established is the key to determining the success or failure of the commercialization of large model companies. As the most leading large-scale model company at present, OpenAI has accumulated strong financial strength and infrastructure support, and has advantages in computing power. This DevDay also demonstrated its ambition to rapidly expand its user base and increase market share by building a developer ecosystem and application platform. This will have a relatively large impact on the entire AI ecosystem and entrepreneurial landscape. We share our thoughts from the aspects that we are most concerned about, including the open-source/closed-source of the large model layer, the agent framework and developer tools of the application layer, and the vector database that we have been discussing.
Large model layer: open source vs closed source
The open source model is fundamentally different from open source software in that many developers contribute very limited to the former. The open source of basic software is that developers can contribute code to polish the software more and more perfectly. However, the open-source model is open to use, and developers can make some peripheral tools, but they cannot contribute to the model itself (from data to algorithms) and cannot change the model. In this way, it is impossible to establish a closed-loop feedback of user data, and there is no ecological effect of open source software.
Closed-source models have always been ahead of open-source models, and OpenAI has made this gap even wider.
- The core resources that affect the development of large models are computing power and scale, and open source models are at a disadvantage. The disadvantage of computing power is mainly that there is not enough capital investment, and OpenAI's efforts in the developer ecosystem and application platform allow it to occupy more users, landing scenarios, data and market share. As a result, OpenAI will always be ahead of open-source models in terms of performance. Even from the current situation, the performance gap between the open-source model and OpenAI is still quite large, and reasoning about the same thing, the former has more illusions and worse dismantling of deterministic things. The release of OpenAI's new model will only make this gap even bigger.
- OpenAI's cost drop and enterprise version support will further eat up the market space for open source models. Although the open-source model cannot match the closed-source model in terms of performance, it is "cheap" and "secure" (easy to fine-tune and deploy privately). But this DevDay, we saw that OpenAI's costs are falling rapidly. In the future, with the improvement of OpenAI's technology and the expansion of scale, the cost of closed-source models will be further reduced. From the perspective of data security, OpenAI cooperates with Microsoft Azure to ensure the data security of enterprise-level customers. Overseas, many large and medium-sized enterprises have their data on Azure, and they have trust in Azure's data protection. OpenAI's cost reduction and data security protection will further squeeze the living space of open-source models.
So, where is the future of open source models? We think it's small-scale, customized. In the future, the general model will become the infrastructure of water, electricity and coal, and the best one or two will win and occupy the vast majority of the market. The open-source model can be applied to provide customized services for some specific domains/vertical scenarios.
- Small, open-source models have application value. The main scenario of the open source model is that enterprises deploy it on-premise for data security. In this scenario, cost is very critical - few companies will have a large number of graphics cards to host the model, and the large open source model is not cost-effective. The Llama2 7B model can be run on a single card, which is the most cost-effective and widely used for enterprise use scenarios.
- Customized DIY. Closed-source models are a general-purpose infrastructure, and they are very difficult to customize (unless they are just simple customizations, such as the GPT builder demo on DevDay, which involves complex processes). In specific scenarios, it is more flexible to use the open source model to do some customization.
We have observed that some startups generally use OpenAI GPT for verification first, and once the verification is successful, they use the open-source model to train a smaller model to reach a similar level of GPT in vertical tasks. Open source and closed source will definitely coexist to serve different scenarios and users.
Application layer: AI agent
AI Agent is the most popular entrepreneurial direction in the AI field in the second half of this year, and a large number of related startups have received financing. From the middle of the year to the present, there are not a few AI agent frameworks that have exploded on GitHub - from the original AutoGPT to the most popular AutoGen recently. The launch of the Assitant API will undoubtedly drop a bombshell on the current lively market, which may reshape the landscape of the AI agent field.
Specifically, the current startups in the field of LLM-based AI agents can be broadly divided into the following two categories:
- The middle layer infra: It mainly provides a practical and reusable agent framework, reduces the complexity of developing agents, and provides mechanism design for agent cooperation. It mainly innovates from modularity, adaptability, collaboration and other aspects.
- Vertical Agent: Drill down into a vertical field, understand the workflow of experts in that field, design Copilot products with agent ideas, and make agent ideas more controllable through user intervention. Quickly form a PMF and start accumulating user data.
With the launch of the Assitant API, we believe that a large number of Agent framework companies will lose their value of existence, and developers will move to the official framework of OpenAI due to ecological convenience and other reasons. Vertical Agent startups will not be affected much, and the core reason is still the "data barrier".
We have done a lot of research on the implementation of AI Agent in specific production scenarios, and found that in vertical fields, the establishment of corresponding "world models" is the core key to making AI Agents. To understand the current task and predict the future scenario, the agent needs to go beyond simple text learning and gain in-depth access to domain knowledge, domain-related private data, and the "process data" of related tasks (how domain experts break down tasks and produce results). These data are difficult to obtain with large models, especially "process data", and many of them even only exist in the brains of experts in the corresponding positions. This requires the relevant companies to do a lot of work to collect, organize, and understand the workflow of the specific business, etc., which is a rather complex system engineering. Especially in the fields of law, medical care, finance and other fields with complex data and high professionalism, it is not possible to solve the problem by "large model shell". Therefore, once a company with a vertical agent can establish and master the "world model" of these vertical industries, it will also have a strong competitive barrier in this uncertain era. We remain very optimistic about the future prospects of these startups.
Choice of entrepreneurial direction
The changes in the entrepreneurial landscape by AI Agent can be extended to another topic: in today's rapid development of the base model, what kind of entrepreneurial direction is relatively safe to choose? As analyzed above, the core value barrier construction for startups still comes from the accumulation of domain expertise/private data/industry world models, as well as the occupation of original customer relationships, channels, and workflow scenarios (with the latter, it is naturally relatively easier to obtain the former). Such companies are not easily overwhelmed by large-scale model capability upgrades/ecosystem expansion, but may instead benefit from the dividends of large-scale model ecosystem expansion. But startups that don't have access to either the big model or the private data/world model, like many companies that make 2D developer tools, face much more uncertainty.
Vector Database &; Memory
AI Memory is one of the issues that plagues a large number of developers. Because of the limited length of Context, the usual solution at this stage is to use vector databases for embedding retrieval, and a number of vector database entrepreneurship projects have emerged. We have observed that a large number of developers have mixed reviews of vector databases, and the biggest problem is that in many cases, accuracy is not guaranteed. In fact, before the era of large models, vector databases were mainly used for recall and image retrieval. In the past, vector databases were more suitable for long-tail matching, and it was usually inefficient to do those matches that were very similar in real meaning. That's why many developers criticize the accuracy of vector databases.
This time, OpenAI provides its own solution for Memory: 128K Context and Assistant API retrieval. With the opening of these two functions, we believe that some small-scale data stores will no longer need a vector database, and can be placed directly in the context, or use the retrival provided by the Assistant API to further optimize the development experience. This means that the use cases of vector databases will be greatly reduced.