Wang Sheng, partner of Inno: In the past three years, we have focused on investing in AIGC infrastructure, and there is no good target on the application side

Launched by Tencent News, the "AI Future Directs North" invites global industry experts, entrepreneurs and investors to discuss technological development, business models, application scenarios, and governance challenges in the AI field.

Text / Tencent Technology Zhou Xiaoyan

The advent of ChatGPT resembles the birth of the iPhone in 2007: because the infrastructure is not mature enough, it is difficult to imagine the future application product portrait. But the experience of the iPhone setting off a wave of mobile Internet entrepreneurship has made today's investors and entrepreneurs ready to prepare for the arrival of a new "iPhone moment".

Since the emergence of GPT-3 in 2020, Wang Sheng, partner of Inno Angel Fund, has been paying attention to the opportunities of generative pre-training models, and has done a lot of related thinking and exploration during this period, and he has divided the investment of a new generation of AI into three levels from bottom to top: AI Infra, large models and applications.

He believes that there are still places that need to be supplemented at the level of computing power and model training framework, and we are in the stage of improving the infrastructure required for AI training and operation (inference), and there is no good investment target on the application side. You can build the infrastructure while waiting for new application opportunities and business models to emerge.

In the large model section, Wang Sheng believes that the imagination of "AI for Science" is very large, and in areas such as aerospace, warships and even architectural and semiconductor design, AI may be able to replace the calculation and solving capabilities of top scientists.

Core Ideas:

1. The core problem of small models is that they cannot be generalized, it is too "stupid", and now advocating small models is nothing more than 2 points:

First, large models are expensive, but this is a cost problem, and this can be solved through industrialization and scale in the future, and the core is how to meet user needs;

Second, in some scenarios, we may not be able to use large models, such as IoT devices with high real-time requirements or single functions, such as smart cameras and smart noise-canceling headphones, and small models in these sectors do have certain needs, but the mainstream paradigm will not be small models.

2. Before the emergence of mobile Internet innovation, all industries were just "big guys" of the original business to do a migration and change the traffic entrance. In the AI era, too, the underlying Infra is perfected, the tool chain is perfect, the development environment is perfect, and then entrepreneurs emerge, and then business transplantation. If you only rely on transplantation, this era will not generate any real investment or entrepreneurial opportunities, and only when the "transplantation" stage passes, will we truly enter the era of model innovation.

3. We are now in the era of AI Infra, traditional businesses are gradually migrating to AI, in this era there may not be too good application-side investment targets, the main investment opportunity is still in infrastructure. Infrastructure investment is a long-term process, but the most important window is in the current era, and the later investment window may migrate to sectors such as business model innovation.

4, the "AI for Science" section is very noteworthy, the most important of this sector is "scientific computing", but mainly Microsoft and DeepMind are doing, startups may not have much opportunity. For example, if we want to design a large aircraft, we need a lot of aerodynamic research, and we currently need to rely on extremely top scientists to design and solve and demonstrate. If AI were to do this, it could solve many problems, including aerospace, warships and even architectural and semiconductor design.

5. The challenge of large model + automatic driving is very large, and the biggest problem is that its tolerance for errors is too low. It doesn't matter if ChatGPT says a few nonsense, but when used on autonomous driving, even one percent or one ten-thousandth of a mistake can cause great safety problems.

Partner of Inno Angel Fund Wang Sheng

The following is a summary of the interviews:

Tencent Technology: In the past six months, because of the high attention in the AIGC field, the investment atmosphere in the primary market is also very "volume", everyone is looking for various investment opportunities, what is the mentality of the investor group during this period?

Wang Sheng: The vast majority of investors are more anxious, but it is not entirely because of the big model, and now the entire economic environment and investment and financing environment are more anxious. The overall financial market environment is not conducive to capital flow, while poor liquidity is not conducive to project exit, thus forming a bad cycle: no exit will affect fundraising, and affecting fundraising will affect investment.

In the investor group, compared with RMB funds, US dollar fund investors will be more "impulsive". In the past, due to policy reasons, new energy or hard technology startup teams were afraid to ask for US dollar money, but AI is a good investment window for US dollar VCs, after all, traditional RMB funds are difficult to withstand the long-term investment requirements of AI startups.

Tencent Technology: In this wave of enthusiasm, what is the investment mentality of Inno Fund?

Wang Sheng: We are relatively calm because we have been deeply engaged in the field of large models for a long time. In June last year, we invested in Shenyan Technology, a pre-trained model technology service provider, whose main goal is to use NLP technology, especially large model (LLM) technology, to serve information processing in the whole process.

If we go back further, in fact, in the second half of the year before last (2021), we were very sure internally that the future is a world of large models, and there are not many opportunities for small models.

Of course, there is still controversy about who has more opportunities for large models or small models. In some specific scenarios with high security requirements, organizations such as governments and enterprises do need to deploy some private models, but this does not represent the development direction of the overall market. I think the mainstream direction of the future is in the world of big models and no small models. Like cloud services, the big model is the infrastructure for a new generation of Internet businesses.

Originally, we expected that the investment enthusiasm of large models would slowly heat up in the past two years, but Open AI suddenly released ChatGPT, which suddenly ignited this track, and suddenly many large model teams emerged, and these teams did not exist when we first looked at this track.

Tencent Technology: The year before last, you discovered that large models are the trend, and small models have little opportunity, what experiments did you go through at that time to reach this conclusion?

Wang Sheng: The core problem of the small model is that it cannot be generalized, it is too "stupid", for example, today the small model can recognize Nongfu Spring, tomorrow replace the Nongfu Spring bottle with blue, it may not recognize. Now the trumpet of small models is nothing more than 2 points:

First, large models are expensive, but this is a cost problem, and this can be solved through industrialization and scale in the future, and the core is how to meet user needs.

Second, in some scenarios, we may not be able to use large models, such as IoT devices with relatively high security requirements, or IoT devices with single functions, such as smart cameras and smart noise-canceling headphones, which have too high requirements for real-time performance and large models cannot run in. From this point of view, small models do have certain needs, but the mainstream paradigm will not be small models.

The big model will become the mainstream, of which the language big model is the crown jewel of artificial intelligence, after all, most of human knowledge is carried in the form of language and writing, and Wittgenstein said that the boundary of human language is the boundary of our thinking. In addition to language models, we also pay great attention to other modal AI models, such as: images, videos, 3D, and so on.

From the second half of 2021, the big model is defined by us as a very important investment direction, and we look for projects in this direction. However, we have also encountered great challenges: those that really invest in making big models, such as IDEA Research Institute, Huawei, and Baidu, cannot be invested in; And there is no team in the market that can make a big model, so there is nowhere to make it.

Tencent Technology: But you invested in Shenyan Technology last year, how did you find them at that time?

Wang Sheng: Inno set up a scholarship in the Department of Computer Science of Tsinghua University, and in 2021, Yifanchao won the first place in this scholarship, and we also began to pay attention to what their team did, which just met the criteria of the project we wanted, so we invested in it.

In 2021, in the multimodal direction, we have contacted multiple teams of Wensheng Tu, but we have not made an investment, mainly because we have not seen the conversion path of commercialization. In the field of literary diagrams, we are very concerned about controllability:

First, can the model be well controlled? Including some precise layout control, such as I want to generate a diagram, I need to have a table in the middle, a person standing next to it, and the arrangement and control of many subjects, etc., whether the model can accurately implement and control the requirements is very critical;

Secondly, the continuity of the model is very important, it is also called "concept definition", such as the first time to ask the machine to draw Iron Man, the second time to draw Iron Man, whether the machine can define the concept of "Iron Man" is very critical, and the problem of secondary creation (Inpainting), such as the first drawing is not satisfactory, and it needs to be modified on the basis.

During that time, we looked at a lot of projects, but they also had a hard time solving both problems.

By February 2023, these problems have been solved by ControlNet, and its emergence is a milestone module, and the controllability of the entire Wensheng graph has been greatly improved.

Tencent Technology: The emergence of ControlNet has indeed greatly improved the productivity of the entire Wensheng graph field, but how can domestic projects compete with star tools such as Midhourney that entered China?

Wang Sheng: Yes, we do face the next question: can we vote? How to vote? Midjourney is competitive, and Stable Diffusion is open source and free. In addition, many entrepreneurs may not have the courage to do this direction, many projects are nothing more than Stable Diffusion combined with a certain industry to make a tool chain, few people will train a general literate graph model. Even if some projects do, their valuations are too high for early-stage institutional investment.

Tencent Technology: Large models are more expensive and valuations are also very high, so some investors look at some vertical models with scarce data, what scenarios and opportunities do you think are in this segment?

Wang Sheng: Many people emphasize that the big model gets common knowledge and it has no private data, but I don't think it's important at all.

First, the knowledge obtained by the vertical model, the large model can get, there are not many things that are purely private by an individual and not owned by others, which is nothing more than the order in which everyone solves the problems.

Second, after the large model has enough knowledge, a lot of small knowledge is not needed, Open AI insiders have mentioned that when they train GPT-4, they deliberately leave a lot of blank knowledge domains (Knowledge Domain), but during the test, they found that even if there is no knowledge of the field of data being fed, GPT-4 can know. Human knowledge correlation is too strong, GPT-4 does not need data feeding to learn this knowledge at all, so it is difficult to count on that rare data to form core competitiveness.

Tencent Technology: Vertical models are not worth investing, so how should large models be invested? Has this round of big model cakes been divided? If you want to enter the market now, what other investment and entrepreneurial opportunities are there in the AIGC space?

Wang Sheng: After the birth of any new industrial technology paradigm, there will be a development cycle, such as the arrival of the "iPhone moment" has a lot of technology accumulation, chip integration and computing power, touch screen, 4G/5G communication, etc.

When the iPhone appeared on the first day, it was difficult for users to understand how much it could do, so the iPhone appeared in 2007, but WeChat appeared in 2011, Didi in 2012, Pinduoduo in 2015, and Douyin in 2016. From 2007 to 2016, there was a 10-year gap in between.

Now the "AI moment" is coming, and it represents a new technological paradigm, but not much it can do in the early stages. The threshold for consumers is high, the infrastructure is not perfect, and the user's understanding of it is much worse. Perhaps the innovation of technology paradigm will allow AI Infra to start first, such as when the cost of chips becomes lower, development tools are more perfect, and there are enough terminals, AI will burst out more opportunity points.

In the first stage of AI development, because everyone's imagination is still very limited, what can be done may be to transplant all the old industries.

Tencent Technology: However, relying on "transplantation" or "migration" alone, it is difficult to change the pattern of various segments of the existing mobile Internet.

Wang Sheng: Before the emergence of mobile Internet innovation, all industries were just "big guys" of the original business to do a migration and change the traffic entrance. In the AI era, too, the underlying Infra is perfected, the tool chain is perfect, the development environment is perfect, and then entrepreneurs emerge, and then business transplantation.

In fact, if you only rely on transplantation, this era will not produce any real investment or entrepreneurial opportunities, only when the "transplantation" stage passes, will it really enter the era of model innovation, which creates new user needs, but this is something we can't see yet, it still takes time and needs future entrepreneurs to create.

When this cycle passes, it will enter the "involution" stage, just like the development of the mobile Internet has entered the end in 2017, it is difficult to produce real innovation, and there will be no new business model, everyone rolls content, such as short videos, live broadcasts, and goods.

We are now in the era of AI Infra, in the process of gradually migrating traditional businesses to AI, in this era may not have a good investment target, we will continue to invest in infrastructure in the next 3 years. Of course, infrastructure investment is a long-term process, but the most important window is in the current era, and the later investment window may migrate to sectors such as business model innovation.

Tencent Technology: In addition to AI Infra, how do you plan to lay out the model layer and the application layer?

Wang Sheng: In the AI Infra sector, we invest in lossless networks and high-speed optical communications, including storage-computing integration, chiplets, etc., and we have also invested in optical chips and quantum directions. Recently, we just invested in an AI chiplet supplier "Primary Semiconductor", which can provide computing power for multimodal large models by improving energy efficiency and reducing costs.

On a higher level, we are also focusing on MLOps, AI development frameworks, AI compilers, etc.

Further up the large model level, in addition to the language big model, we will also look at the opportunities of multimodal large models and vertical knowledge domains:

In the language model sector, we invested in Shenyan Technology;

In the multimodal sector, we have invested in KIRI Innovation, a 3D scanning reconstruction technology company, and they are currently working on 3D artificial intelligence generative models, direct text to 3D; We also invested in the big model of virtual human action behavior, which belongs to Text to Motion, which is an end-to-end generative AI virtual human technology company, which can effectively improve the current situation of game characters relying too much on action libraries and expression libraries, and relying on the "library" model will lead to the problem of lack of change in character expression and rigidity.

There have been a lot of small model companies in the CV space in the past, and in April we saw that Meta's SAM model showed very strong generalization capabilities, and we are also looking at investment opportunities for real CV large models.

In the field of image and video generation, because SD open source, MJ and Runway are very powerful, we see that a large number of domestic entrepreneurs are applying based on SD, rather than self-developed models and these few rigid. We are very much looking forward to the emergence of courageous and powerful entrepreneurial teams in China.

Another investment direction for large models is AI for Science, which is a relatively early and controversial field, and the cognitive threshold for entrepreneurship and investment is very high.

The AI for Science section is divided into three paradigms: scientific computing large model, vertical domain knowledge large model, and vertical domain generative AI.

Among them, scientific computing is the lowest paradigm, it can use AI to help scientists do calculations, logically it can solve all scientific research problems, but mainly Microsoft and DeepMind are doing, startups may not have much opportunity. For example, if we want to design a large aircraft, we need a lot of aerodynamic research, and we currently need to rely on extremely top scientists to design and solve and demonstrate. If AI were to do this, it could solve many problems, including aerospace, warships and even architectural and semiconductor design.

The second paradigm is the vertical domain knowledge big model, which is also the paradigm that we are most concerned about and will invest in soon, which is based on the LLM large language model, plus a large number of scientific papers and related experimental data for training, hoping to emerge similar to ChatGPT and GPT-4 thinking chains and reasoning capabilities, and even have a certain degree of theory of mind (mind) ability.

The third paradigm is the vertical domain generative AI, compared with the first two paradigms, although it ran earlier, but its development degree is not more mature than the first two paradigms, it learns a lot of knowledge and data about molecular structure to achieve speculation on DNA structure, molecular structure, and at the same time do some analysis and speculation of forces between molecules. It can be used in pharmaceutical, protein synthesis, especially in macromolecular biorelated applications. But it is also more controversial, and although it has shown greater efficiency and can predict new structures faster than humans, it has not been experimentally proven to be effective.

Regardless of the paradigm, AI for Science can theoretically solve all scientific problems, such as renewable energy problems and climate problems closely related to human life, but the most important application direction of the entire AI for Science is two: medical pharmaceuticals and materials, because these two fields have the strongest ability to make money.

Tencent Technology: These directions are still focused on the model level, but at this stage, everyone is actually more concerned about the opportunities at the application level, Inno has observed the opportunities of AIGC since 2020, what imagination is there at this stage of the application layer?

Wang Sheng: It is very difficult to invest in terms of application.

We don't really want to assume this, I don't think it's yet to the stage of applied innovation, no one can assume a completely new scenario, only entrepreneurs make it, investors can see it, we need to wait.

AI to the transformation of traditional industries is actually not an entrepreneur's "dish", even if someone can do it, this person must be extremely influential in the traditional field, or especially understand the industry, have very strong resources, these may be very special fields. In other fields, such as office scenarios, such as doing smart PPT, only Microsoft can do it; Doing smart home and architectural design can only be done by AutoDesk, which does not make much sense for entrepreneurs.

Tencent Technology: Even for many companies trying to transform traditional industries with AI, it is a very difficult process to open up various data internally.

Wang Sheng: Yes, it is very difficult, and it may take a lot of patience to wait.

Tencent Technology: You once said that the era of AI 3.0 is the era of "embodied intelligence", which can drive humanoid robots with large models, and many capabilities of automatic driving and humanoid robots are connected.

Wang Sheng: The challenge of large model + autonomous driving is very big, and I think the biggest problem is that its tolerance for errors is too low. It doesn't matter if ChatGPT says a few nonsense, but when used on autonomous driving, even one percent or one ten-thousandth of a mistake can cause great safety problems.

We may not be able to say that this AI breakthrough will definitely make automatic driving land, but the introduction of intelligent cockpits into large models will definitely improve the user experience, and of course, it will also increase the cost of users using the car.

In addition, I think it will have a relatively large impact on the entire autonomous driving industry, such as how much computing power should be given in the car? Is this computing power provided by NVIDIA, or is it provided by specialized automotive-grade AI inference chips?

Tencent Technology: Is this essentially because large models are good at solving the problem of public attributes, but it is difficult to solve some very special corner cases (corner scenes) of autonomous driving?

Wang Sheng: Large models may not be able to effectively identify these corner cases, but the intelligence level of large models will increase significantly, and even if it cannot recognize them, there may be ways to deal with them. For example, if we hit an object on the road that you've never seen before, a human driver knows how to handle these situations, and a large model that can't recognize it, but it has a high level of intelligence and may be able to handle these situations.

Tencent Technology: If you insist on training large models to recognize various corner cases, maybe it can infinitely approach the intelligence and recognition level of human drivers?

Wang Sheng: Yes, let's not underestimate the future, it may be my scientific and technological optimism, I believe that technological progress can solve many problems. Many people are more likely to overestimate short-term trends and greatly underestimate long-term trends, including the investment boom of large models, and many companies with high valuations cannot be invested, in fact, this is highly optimistic about the short term, not rational.

Tencent Technology: You once said that among the Tsinghua entrepreneurs who make big models, the ones who can fight Wang Xiaochuan and Shenyan Technology the most are the most important.

Wang Sheng: The core is still to look at the team, the most important thing is the "faith" of the team, everyone has seen the big model, some people believe in AGI, some people do not believe. For example, Transformer is made by Google, but Google does not have such great faith, they say that Encoder is good, Decoder is also good, and then make an encoder + decoder T5 out, but only Open AI is desperate to explore in the direction of GPT.

After we invested in Shenyan Technology, we immediately arranged for Shenyan to do the next round of financing, and took them to meet dozens of institutions, but none of them were willing to invest. Later, the investor who pulled Sequoia and Yu Fanchao ate a beef hot pot together, and after eating, Sequoia agreed to invest. Now, there are too many investment institutions that want to squeeze in, so the difference between the results caused by believing and not believing is still very large.

No one is sure whether AI can emerge scientific research capabilities, and we can't judge at what point in time, how much ability AI can emerge, and how much problem it can solve, but we just "believe". We've talked to a lot of scientists about AI for Science, some scientists are excited, some think it's nothing, and even if you understand it, you can't judge it.

About who can do this, it depends on the investor himself to believe it or not, we can find many reasons, such as making a large model requires past experience, because there are many challenges to be faced, such as data, engineering, algorithms, distributed training, etc., in this regard, Shen Yan technology has great advantages.

If you think that making a large model requires a strong financing ability, then Wang Huiwen may be better; If you think that scientific research and scientists are more important, then you should vote for Teacher Tang Jie; If you think that doing engineering is more important, you can invest in Zhou Bowen.

So the core still depends on what you believe, sometimes not because you really believe, but because you want to make yourself believe.

Tencent Technology: So who to vote for or who not to vote for may also depend on whether the "aura" fits or not.

Wang Sheng: Yes, we invested in Shenyan Technology last year, and this year we also talked about related projects in the market, and after the talk, it strengthened our confidence in continuously blessing Shenyan Technology. Because you've invested in Shenyan technology in the past, you'll know exactly where their abilities lie and it will strengthen your faith.

But for new entrants, they don't take sides, they will feel that entrepreneurs are more important, and may invest in Wang Huiwen or Wang Xiaochuan. In general, there is no core standard for this, and each has its own reasons.

Tencent Technology: Have you talked to Wang Xiaochuan and Wang Huiwen, why didn't you invest in them?

Wang Sheng: The main thing is that they are too expensive, and for early-stage funds, $500 million or $1 billion is too expensive.

Tencent Technology: How many investors are looking at AIGC?

Wang Sheng: We have an AIGC group with three investors, but we require investors in every track to understand AI, because AI will change all industries. Whether you look at the direction of materials, or life technology, or chips, semiconductors, everyone must think about AI.

Tencent Technology: When did you set up this group?

Wang Sheng: The year before last (2021), I was mainly in charge of this group.

Tencent Technology: How does this group work?

Wang Sheng: We do not emphasize KPIs, mainly rely on everyone's self-drive, investors are basically looking at projects day and night, our requirements for investors are, first of all, we must love this industry; Secondly, it is necessary to be professional, so that you can find good projects; The third is to look at the investment performance, in the short term, if you invest in a good project, the project grows rapidly, investors also have a great sense of satisfaction, in the long run, the final income generated by a good project will also give everyone a good distribution incentive mechanism.

Tencent Technology: How many projects has your AIGC team invested in so far? What is the approximate size of a single investment?

Wang Sheng: In this wave of big model and generative AI investment, Inno is one of the institutions with the largest number of contact projects and the highest quality, but really good projects are very scarce, and the number of our investments can be counted on one hand. The number of investments is also related to the different strategies of each fund.

We want every project to be accurate, for example, we have seen some projects, we did not invest, but Sequoia Capital did. It's not that we're stricter than them, but their fund is bigger, with seeds, VC, growth, and even secondary market business segments, they are more afraid of "missing out", we are more afraid of "investing in the wrong place", and maybe we will miss some projects because of this.

We have two types of funds, one is for the traditional angel type of investment in the past, a single investment of about 10 million, and now there are some early projects that have become very expensive, so we have made a larger fund, a single investment of about 20 or 30 million.