When open source meets large models, what kind of changes will occur?

The emergence of ChatGPT has greatly stimulated the industry's interest in generative AI, and many companies have started the journey of R&D and application of AI programs, and the pace of innovation is accelerating. Over the past 30 years, the rapid innovation of open source technology has greatly reduced IT costs and innovation thresholds, and has become a common choice for IT enterprises. So, when a new large model meets a mature open source technology, what kind of changes will it produce?

Red Hat has been leading the open source trend from offering an open enterprise Linux platform through RHEL in the early 21st century to pushing containers and Kubernetes as the foundation for open hybrid cloud and cloud-native computing with OpenShift. Recently, Red Hat announced two major developments: the launch of Red Hat Enterprise Linux AI (RHEL AI), which introduces new enhancements to OpenShift AI to better integrate Linux, OpenShift, and RHEL with AI and large models.

Democratizing open-source generative AI innovation with RHEL AI

Red Hat Enterprise Linux AI (RHEL AI) is a foundational model platform that makes it easier to develop, test, and deploy generative artificial intelligence (GenAI) models. RHEL AI integrates IBM Research's open-source licensed Granite large language model (LLM) family, InstructLab model alignment tool based on the Large-Scale Conversational Bot Alignment (LAB) methodology, and a community-driven model development approach implemented through the InstructLab project. The solution is packaged as an optimized, bootable RHEL image for deploying a single server in a hybrid cloud environment and is integrated into OpenShift AI.

Red Hat believes that implementing an AI strategy is much more than choosing a model. Enterprises need expertise to adapt the right model for a specific scenario and deal with the cost challenges of AI implementation. To lower the barrier to entry for AI innovation, companies need to expand the range of people involved in AI projects while controlling the associated costs.

IBM Research developed LAB technology, a model alignment method that leverages classification to guide synthetic data generation and innovation in a multi-stage tuning framework. This approach makes the development of AI models more open and accessible by reducing reliance on expensive human annotations and proprietary models. At a time when AI is advancing by leaps and bounds, Red Hat decided to adopt IBM's LAB.

Matt Hicks, president and CEO of Red Hat, recalls that at a meeting at IBM Research, IBM introduced new technologies such as LAB, a way to address the challenges customers face in the fine-tuning process, using synthetic data augmentation techniques to make chunks small enough to collaborate. Matt Hicks was so impressed with this technology that he quickly realized that Red Hat could build a community around LAB to bring global innovation into a realm where massive amounts of data were gathered.

IBM and Red Hat decided to launch the InstructLab open source community, with the goal of putting the power of LLM development in the hands of developers just like any other open source project by simplifying the process of creating, building, and contributing to LLMs.

RHEL AI uses an innovative approach to open AI, combining the enterprise-ready InstructLab project and the Granite language and code model with the world's leading enterprise-grade Linux platform to simplify deployment in hybrid infrastructure environments and build a foundational model platform that enables open-source licensed generative AI models to be used by enterprises.

As organizations experiment and adapt new AI models on RHEL AI, they can scale their workflows with Red Hat OpenShift AI, which will include RHEL AI and leverage OpenShift's Kubernetes engine to train and deploy AI models at scale, as well as OpenShift AI's integrated MLOps capabilities to manage model lifecycles. RHEL AI, once available in OpenShift AI, will bring additional capabilities for enterprise AI development, data management, model governance, and price/performance improvements.

"Generative AI represents a revolutionary leap forward for enterprises, but it requires them to actually deploy and use AI models for their specific business needs. By combining the broad application of Red Hat OpenShift AI, the RHEL AI and InstructLab projects aim to alleviate the multiple challenges of generative AI in hybrid cloud, from the limitations of data science skills to the huge resource requirements, while facilitating enterprise deployment and driving innovation in upstream communities. Ashesh Badani, senior vice president and chief product officer at Red Hat, concluded.

OpenShift AI enhances predictive and generative AI flexibility in hybrid cloud

Red Hat OpenShift AI is an open hybrid AI and machine learning platform built on Red Hat OpenShift that enables organizations to create and deliver AI-powered applications at scale in hybrid cloud environments.

Red Hat's AI strategy supports flexibility across hybrid cloud environments, the ability to augment pre-trained or curated foundational models based on customer data, and the freedom to enable multiple hardware and software accelerators. Red Hat OpenShift AI introduces new enhancements to meet these needs, including access to the latest AI/ML innovations and support from a vast ecosystem of AI-focused partners. The latest version of the platform, OpenShift AI 2.9, provides edge model services, enhanced model services, support for distributed workloads with Ray, improved model development, model monitoring and visualization, new accelerator profiles, and more.

As AI models move from experimental to production, customers face a number of challenges, including increased hardware costs, data privacy concerns, and a lack of trust when sharing data with SaaS-based models. These challenges are exacerbated by the rapid changes in generative artificial intelligence (GenAI), and many businesses are actively working to build a reliable core AI platform that can run on-premise or in the cloud.

To successfully leverage AI, enterprises need to modernize many existing applications and data environments, remove barriers between existing systems and storage platforms, improve infrastructure sustainability, and carefully choose where to deploy different workloads between the cloud, data center, and edge, according to IDC's Future of Digital Infrastructure, 2024: AI-Ready Platforms, Operating Models, and Governance Survey. For Red Hat, this means that AI platforms must be flexible enough to allow organizations to adapt to changes in demand and resources as they adopt AI.

The latest advancements in OpenShift AI underscore Red Hat's vision for AI: from the underlying hardware to services and tools, such as Jupyter and PyTorch, Red Hat supports customers to accelerate innovation and productivity. Through this more flexible, scalable, and adaptable open source platform, we can help enterprises apply AI to their daily business operations.

"To meet the demands of large-scale enterprise AI, Red Hat offers Red Hat OpenShift AI. This solution enables IT leaders to deploy intelligent applications in a variety of locations across the hybrid cloud, while scaling and fine-tuning operations and models based on demand to support the needs of real-world production applications and services. Ashesh Badani, chief product officer and senior vice president at Red Hat.

Red Hat embraces generative AI

"In the past year, AI has improved 100 times or even 1,000 times. Back a year ago, the technology had shown great potential. At that time, models like ChatGPT had just emerged and had already demonstrated the power of language models in their early stages. Today, progress is evident. Recalling the development of artificial intelligence this year, Matt Hicks said.

At the helm, Matt Hicks is responsible for keeping a close eye on cutting-edge technologies, making trend predictions, and shaping Red Hat's growth strategy. "Whether we can give full play to the influence of open source in this field is a question I have been thinking about."

According to Matt Hicks, there is no doubt that as models get smaller, they will train faster and faster, allowing them to perform more functions more efficiently. This rapidly evolving dynamics will continue to accelerate, significantly impacting the way companies operate and adapt to new technologies. After a year of development, the industry has moved beyond the experimental proof stage of large models to give birth to incredible innovations. That's why Red Hat's focus is to stay ahead of the curve and make sure these advancements translate into real opportunities for customers. "It reminds me of the late '90s when I felt about the potential of Linux, when it was still in its infancy, but my gut told you that big changes were coming."

In Matt Hicks' view, there will be a lot of intersection between traditional IT technology and artificial intelligence. "It's one thing to be able to run a model that's already built, it's even more powerful to train the model to customize your own solution, and then you have to apply it in practice." In the case of RHEL and RHEL AI, as well as OpenShift and OpenShift AI, the parallels are as follows: RHEL is used to run all applications that can run on Linux, while RHEL AI is used to run the AI part of a large language model that can be trained and customized; OpenShift is used to manage all applications that can run across clusters in a distributed manner on RHEL, while OpenShift AI manages a range of models in the same way, efficiently splitting training, consumption, and service delivery. Because AI doesn't exist in isolation, there are many cross-links and integrations between traditional setups and AI applications.

While Red Hat bundles AI models with Linux, Linux will continue to play a vital role in the IT world dominated by big models. "Although the way we use Linux is evolving, the core software stack that supports the operation of large language models is heavily dependent on Linux." According to Matt Hicks, Linux optimizes the integration and performance of these software, and acts as a coordinator between GPUs, CPUs, and emerging hardware types.

The development of AI is inseparable from cloud infrastructure, and the industry is quietly changing in the choice of cloud. Many customers have tried to fine-tune and train with smaller models, often with poor results, so they have turned to "omniscient models" – often running in the public cloud with more than 1 trillion parameters. While these models accomplish many tasks out of the box, they are very expensive to run and train. When constrained by the cloud environment, it is difficult to cope with many scenarios. For example, running a model on a laptop and the data never leaves the laptop. That's why Red Hat believes that hybrid capabilities are key to realizing the full potential of AI, whether it's in laptops, at the edge of the factory, or in cars.

Matt Hicks concluded by saying that Red Hat is committed to supporting hybrid deployments of small, open-source models, where users can train models in the public cloud or data center and deploy them at the edge. Red Hat's platform supports major GPU providers such as NVIDIA, AMD, and Intel, providing flexibility from the public cloud to the edge. In addition, Red Hat is unique in going beyond Linux and virtualization alone. Red Hat has taken advantage of the situation and expanded its strengths in areas such as Linux and virtualization to include middleware, virtualization, OpenStack, containers, and artificial intelligence. "Red Hat's ability to disrupt itself and keep up with technology trends is what sets Red Hat apart and makes it a valuable partner." Matt Hicks concludes.