On May 15, the 2024 Spring Volcano Engine FORCE Conference was officially held in Beijing. At the meeting, the latest products such as ByteDance's bean bag large model family, Volcano Ark 2.0, AI applications and AI cloud infrastructure were officially released.
President of Volcano Engine, Tan Zhi
Tan Bei, President of Volcano Engine, said that in order to do a good job in the implementation of large models, enterprises face three key challenges: model effect, inference cost, and landing difficulty: not only to have a good basic large model to support and solve complex problems, but also to have low-cost inference services to make large models widely used, and more tools, platforms and applications to help enterprises do a good job in scenario landing.
The bean bag large model debuted with a large amount of use to polish the model
Model effect is the most critical challenge for AI implementation. Tan Bei pointed out that only a large amount of use can polish a good model. At present, the Doubao model processes an average of 120 billion tokens of text and generates 30 million images per day.
In order to help enterprises do a good job in the implementation of large model scenarios, the large bean bag model independently developed by ByteDance will be officially provided through the Volcano Engine, and enterprises can flexibly choose and quickly implement according to their own business scenario needs:
Doubao General Model Pro: ByteDance's self-developed LLM model professional edition, which supports 128k long texts, the full series can be fine-tuned, has stronger comprehensive capabilities such as understanding, generation, and logic, and adapts to rich scenarios such as Q&A, summarization, creation, and classification;
ByteDance's self-developed LLM model lite, which provides lower token cost and lower latency than the professional version, providing enterprises with flexible and economical model options.
Bean Bao Role-playing Model: Personalized character creation ability, stronger context perception and plot promotion ability, to meet flexible role-playing needs;
Bean Bag Speech Synthesis Model: It provides natural and vivid speech synthesis capabilities, and is good at expressing a variety of emotions and interpreting a variety of scenes;
Bean bag sound replica model: 1:1 cloning of sound can be achieved in 5 seconds, which highly restores the similarity and naturalness of timbre, and supports cross-language transfer of sound;
Doubao speech recognition model: higher accuracy and sensitivity, lower speech recognition delay, and support for correct recognition in multiple languages;
Bean Bao Wensheng Diagram Model: More accurate text comprehension ability, more accurate picture and text matching, more beautiful picture effect, good at the creation of Chinese cultural elements;
Bean Bag · Function call model: provides more accurate function identification and parameter extraction capabilities, which is suitable for complex tool invocation scenarios.
Bean Bag Vectorization Model: Focusing on the use scenarios of vector retrieval, it provides core comprehension capabilities for the LLM knowledge base and supports multiple languages.
Tan Bei introduced that as early as last year, the Doubao large model (formerly known as: Skylark) was launched within ByteDance, and it is also one of the first large models to pass the security filing of the large model service. More than 50 businesses within ByteDance have used a large number of bean bag models for AI innovation, including hundreds of millions of DAU products such as Douyin and Toutiao. Based on the Doubao model of the same name, the Doubao APP ranks first in the number of downloads of the Doubao APP in the Apple App Store and major Android application markets*.
*数据来源:QuestMobile
Jun Zhu, Vice President of Product and Strategy at ByteDance
Zhu Jun, vice president of product and strategy at ByteDance, introduced ByteDance's product concept of building AI native applications based on the bean bag model. He said that technology is evolving rapidly, but the core needs of users have not changed, such as the need for efficient access to information, the need for work efficiency, the need for self-expression, and the need for social entertainment. ByteDance has done a lot of learning and exploration, hoping to find a suitable large-scale model application form, serve users in a natural interactive form, and enter more people's lives and work.
According to Zhu Jun, the Doubao app has 26 million monthly active users and more than 8 million agents have been created.
While serving the inside of Bytes, the bean bag model is also cooperating with many enterprise customers of Volcano Engine to co-create. Tan believes that enterprises need models with different characteristics in different business scenarios to achieve the best results and cost performance. To this end, the Doubao model provides a multi-modal model family, which enterprises can flexibly choose according to their own business scenario requirements to achieve a positive cycle of model use and cost control.
A good model is something that everyone can afford
Tan believes that cost reduction is a key factor in driving large models to fast forward to the "value creation stage". The pricing of the main model of bean bags in the enterprise market is only 0.0008 yuan/1000 tokens, and 0.8 centimeters can process more than 1,500 Chinese characters, which is 99.3% cheaper than the industry.
Taking the pro-32k version of the bean bag general model as an example, the input price of the model inference is only 0.0008 yuan/1000 tokens. The pricing of the model of the same specification on the market is generally 0.12 yuan/1000 tokens, which is 150 times the price of the bean bag model. The price of large models has entered the "centripetal era".
Tan said that the price of large model "volume" will help enterprises accelerate business innovation at a lower cost.
Ark 2.0 upgrade to promote the value creation of large models
Last year, Volcano Engine released a one-stop large-scale model service platform, Volcano Ark, hoping to help enterprises apply various models in an efficient and safe environment through the concept of model-as-a-service. This year, Volcano Engine has comprehensively upgraded the Ark platform, and the upgraded Volcano Ark 2.0 will greatly improve the model effect, core plug-ins, system performance and platform experience, helping enterprises promote the value creation of large models.
In terms of plug-ins and tool chains, Ark 2.0 has upgraded networking plug-ins, providing the same search capabilities as Toutiao Douyin, connecting massive high-quality data in real time, using multi-modal interaction methods such as text, images, and voice, and greatly improving the model's information acquisition ability through industry-leading intent recognition capabilities. Upgrade the content plug-in to provide Toutiao Douyin with the same source of massive content, support multi-modal interaction, provide intent-based vertical content information retrieval, and stronger content timeliness retrieval, helping large models to deeply understand, retrieve and generate content; The knowledge base plug-in is upgraded to provide high-performance retrieval with a scale of tens of billions in milliseconds, a second-level streaming knowledge base index update, and an embedded bean packet vectorization model to improve the relevance and accuracy of searches.
In addition to the above core plug-in upgrades, Volcano Ark 2.0 has also comprehensively upgraded the system carrying capacity, security protection capabilities and algorithm service capabilities. In terms of system carrying capacity, it provides abundant GPU computing resources and super elasticity of minute-level kilocalorie expansion and contraction, ensuring business stability and controllable costs. In terms of security protection, a trusted execution environment and a multi-dimensional security architecture are built through the security sandbox to ensure data security. In addition, Volcano Engine provides professional algorithm team services to help customers unleash the value of unique data and make it easy for enterprise large model applications to land.
Buckle Product Manager Pan Yuyang
In addition to good model effects, convenient plug-ins, and strong system performance, it is also necessary to provide an easy-to-use development platform so that everyone can become a developer of AI applications. Pan Yuyang, product manager of Buckle, introduced that as a new generation of AI application development platform launched by ByteDance, Coze has the advantages of low threshold, personalization, real-time, and multi-modality, and integrates massive AI resources, rich release channels, and one-click custom API services to help fill the last mile from large models to user scenarios.
Volcano Engine has officially released the Buckle Pro. On the basis of the ability of dynamic and flexible orchestration of intelligent twins through visualization, Button Professional Edition further provides enterprise-level SLAs and a variety of advanced features, making AI applications easier to implement and allowing enterprises to focus more on driving business growth through intelligent twin innovation.
Tan said that Volcano Engine is committed to becoming the most open and leading cloud service platform in the AI era. The newly upgraded Volcano Ark 2.0 will help the application of large models through stronger performance, better plug-ins, better services, and secure and trustworthy solutions.
Extensive implementation and practice in many industries have yielded fruitful results
The bean bag model and Volcano Ark 2.0 have been implemented in many industries such as finance, automobiles, smart terminals, e-commerce retail, education and scientific research. China Merchants Bank, Haidilao Hot Pot, Super Orangutan, Ctrip Travel, Feichang Quasi, Liepin and other companies have also built intelligent twins on the button. Fudan University, Zhejiang University and other famous universities have also set up "AI teaching assistants" for courses and experiments.
Gao Xulei, Director of Digital Finance Development Office of China Merchants Bank
As a practitioner of AI transformation in the financial industry, China Merchants Bank is exploring and investing in hundreds of AI service scenarios such as intelligent outbound calls, digital humans, and intelligent data assistants. Gao Xulei, director of the Digital Finance Development Office of China Merchants Bank, said that the large model will bring transformation and upgrading to the bank's entire chain business. Together with Volcano Engine, the two parties will jointly build large-scale model application scenarios and high-performance infrastructure, transform existing systems and products, and give birth to a large number of AI-centered intelligent twin applications.
Li Chengjie, Vice President and Chief Digital Intelligence Officer of Mengniu Group
Li Chengjie, Vice President of Mengniu Group and Chief Digital Intelligence Officer of Mengniu Group, introduced the digital intelligence process of Mengniu Group from three aspects: the evolution of production tools, the evolution of production methods, and the evolution of organizational methods. Mengniu Group is working with Volcano Engine to consolidate the technical foundation and build an intelligent twins matrix such as AI nutrition experts, AI nurserymen, AI data analysts, product innovation assistants, and hot creative assistants to improve the front-end consumer experience and help back-end operators make decisions.
The Intelligent Terminal Large Model Alliance was released
At the meeting, Volcano Engine and OPPO, vivo, Honor, Xiaomi, Samsung, and Asus announced the establishment of a smart terminal large model alliance. Applications such as OPPO's Xiaobu Assistant, YOYO Assistant of Honor MagicBook, Xiaomi's "Xiaoai", and Asus Laptop's Douding AI Assistant have all been connected to the large model service of Volcano Engine.
The Automobile Large Model Ecological Alliance was released
At the same time, Volcano Engine, together with more than 20 manufacturers such as Geely Automobile, Great Wall Motor, Jietu Automobile, Celis and Zhiji Automobile, announced the establishment of an automobile large-scale model ecological alliance, which is committed to exploring a new form of human-vehicle interaction for a new generation and improving the new AI experience of all scenarios in automobiles.
The two major alliances, OPPO and Jietu Auto, also shared their own large-scale model practices:
Wan Yulong, director of OPPO's intelligent assistant department and chief architect of Xiaobu assistant
Wan Yulong, director of OPPO's intelligent assistant department and chief architect of Xiaobu Assistant, shared the application practice of large models of OPPO AI mobile phones, and he said that at present, Xiaobu Assistant has covered 350 million terminal devices and has more than 150 million monthly active users. On the Volcano Ark, OPPO has built a general Q&A, smart little girl, interviewer and other agents through the large model of intent recognition, online summary, personalized character and bean bag, bringing a new AI experience to end users.
Liu Hang, head of digitalization of Jietu Automobile
Liu Hang, head of digitalization of Jietu Auto, said that Jietu Auto hopes to expand its service capabilities and gain insight into the real needs of users through large models, so as to improve sales efficiency and capabilities. Working with Volcano Engine, the two parties have co-created around customer service scenarios, and the intelligent customer service AI Xiaojie created can not only save manpower, but also help customer service personnel quickly obtain information and serve customers. In the future, Jietu Auto also hopes to further realize the intelligent upgrade of cockpit, decision-making and marketing through the large model.
A variety of AI applications have been released and quickly implemented out of the box
In addition to the large-scale model service platform, AI applications are also a key component of Volcano Engine's full-stack AI services.
Last year, Volcano Engine proposed a new model for enterprise digital and intelligent upgrading, the data flywheel, which takes data consumption as the core driving force, so that enterprise data flow can be fully integrated into the business flow and realize the flywheel effect of data assets and business applications. In order to make enterprise data consumption more inclusive and convenient, Volcano Engine Intelligent Data Insight DataWind officially released the AI assistant ChatBI, which allows users to generate indicators through natural language interaction for chart making, data query, and in-depth data analysis.
For marketing scenarios, Volcano Engine has upgraded Intelligent Creation Cloud 2.0 to become a one-stop enterprise content marketing growth solution from content creation, matrix distribution, advertising to data insight. It not only provides multi-modal understanding and generation capabilities, helping to improve the efficiency of video creation by 25 times, but also launches marketing tools such as Douyin topics, POIs, and mini programs, increasing store page exposure by 600%. At present, 3,000+ companies have started AIGC video marketing growth in Volcano Engine.
For sales scenarios, Volcano Engine has released Sales Copilot, a sales AI assistant that can efficiently follow up customer needs and assist sales in answering complex product questions at any time. In addition, it can also simulate different styles of customer rehearsals through role-playing, and provide excellent speech learning to improve the quality of sales communication.
Volcano Engine has been committed to building an open and shared ecosystem, and since the release of the Universal Plan in 2021, it has helped 100+ SaaS companies achieve business growth and achieve nearly 10,000 customer signups. The Doubao Enterprise Service Alliance is a special plan of the Universal Plan for the landing of large models. The Wanyou Plan Doubao Enterprise Service Alliance will provide a package of support plans to help SaaS companies achieve AI upgrades and provide more out-of-the-box AI applications for industry customers.
AI infrastructure has been upgraded to a new dimension, and cloud computing power is reliably guaranteed
In the era of large models, the complexity of cluster management, network performance, and stability are the key challenges of cloud foundation. In order to meet the above challenges, Volcano Engine provides a set of industry-leading solutions for the construction, management, performance optimization, and stability assurance of large clusters based on the resource scale advantages and internal practices of internal and external common pools.
Junjie Lai, vice president of solutions and engineering at NVIDIA
At the conference, Volcano Engine officially released veOmniverse, which provides a new cloud-based development platform for enterprises and developers based on NVIDIA Omniverse, helping enterprises and developers to efficiently design and collaborate around workflows in the interconnected virtual world, and explore and create intelligent and realistic virtual worlds.
In addition to providing customers with high-stability and high-performance cloud infrastructure, Volcano Engine has also released a new hybrid cloud veStack intelligent computing version to meet the needs of on-premise deployment, which has the advantages of 10,000-card cluster networking and 3.2T high-performance lossless network. The ultimate performance supported by end-to-end engineering can achieve 97.78% training acceleration ratio and minute-level fault detection and self-healing. It has multiple heterogeneous capabilities, can adapt to more than 10 kinds of GPUs, and supports mainstream localized GPUs.
In addition to the intensive computing performed by GPUs, complete AI applications also need to continuously reduce costs and increase efficiency of existing CPU loads, ensuring that services can be operated lightly and innovation accelerated. From the four levels of computing, storage, network, and O&M, Volcano Engine fully supports the stable operation of customers' cloud services.
From technological innovation to commercial implementation, large models have provided powerful momentum for AI transformation in various industries. Based on ByteDance's practical precipitation in the field of large models, Volcano Engine polishes good models through large usage, makes large models affordable to everyone with technology-driven extreme cost performance, and accompanies customers to grow together through rich landing experience and algorithm services in various industries. In the future, Volcano Engine's full-stack AI services will continue to help enterprises implement AI transformation and stimulate business growth potential.