Do K8s really match large models? This article throws up a question, but the answer remains to be verified.
K8s ushered in a new audience
Let's say a machine learning researcher reads a research paper and wants to test it in a PyTorch environment using a Python-based GPU. She asked her engineering team to access a Jupyter notebook with two GPUs and all of her libraries.
The engineering team told her it would take three days. They have to get the source of the GPU, create a stack, and then grant access to JupyterHub.
"That's exactly what DevOps experienced 10 years ago," Janakiram, an independent analyst, said in a conversation with KubeCon + CloudNativeCon Europe in March.
"So the whole idea now is, how do we speed up this process and enable enterprise IT to bring the infrastructure to a point where it's readily available to ML researchers, engineers, and developers so that they can quickly turn their ideas into code?"
The new personas reflect the impact of large language models (LLMs) on the cloud-native community and raise questions about identity and Kubernetes roles. Do data scientists even need Kubernetes to put their models into production?
Independent analyst Sanjeev Mohan believes that NVIDIA's inference microservice Nim is a Docker container orchestrated for Kubernetes.
The challenge is that Kubernetes will become deeply data-centric, which is due to the stateful and frequently changing nature of data. Data has never played such an important role in the Kubernetes community. The Kubernetes community has never needed to adapt in this way to the new demands of generative AI, model development, integration, deployment, and management.
Without a standard approach to deploying a data model on Kubernetes, future work will require the community to adapt to the new "data roles" through new hardware integrations and projects.
How does AI make K8s more powerful?
Kubernetes Service LLMs, what can LLMs do for K8s?
But really, what is the role of Kubernetes in AI? The question of data roles has brought this to the forefront. Kubernetes is a control plane – yes, it makes sense. It has been an application architecture for DevOps since 2014.
As a result, one of the questions Mohan asks becomes even more relevant: does K8s serve AI, or does AI serve K8s?
At KubeCon, we've seen a lot of how Kubernetes can be used as a control plane for AI. In NVIDIA's keynote, they discussed the dynamic resource allocation that allocates a portion of the GPU. This saves costs. This is the Kubernetes of artificial intelligence. All of these developments are going well, and Mohan says that we will see more and more Kubernet become the control plane of general AI.
But on the one hand, how can LLMs make K8s more powerful? Mohan asks a very imaginative question.
"I haven't seen much of this yet, and maybe at some future KubeCon we'll start to see a higher level of integration," he said. ”
OpenAI is undoubtedly an ally of Kubernetes, and the company is using Kubernete to launch and scale up experiments.
As a popular AI research lab, OpenAI needed a deep learning infrastructure that would enable experiments to run in the cloud or in its own data center, and be easily scalable. Portability, speed, and cost are the main drivers.
Sudha Raghavan, Oracle's senior vice president at KubeCon, asks how Kubernetes will be the default option for all AI workloads without data scientists and data engineers thinking about how to configure it to make the most efficient use of any hardware GPU?
Raghavan also spoke about a panel discussion at KubeCon where it's easier for people to work on a per-workload basis, where engineers can configure templates out of the box and understand that these are AI workload patterns that haven't yet emerged, and that there are predefined templates.
So, any data scientist who wants to do experiments doesn't have to learn on their own, but can learn what the Cloud Native Computing Foundation has to offer the AI and ML community in its ecosystem.
Arun Gupta, vice president and general manager of Intel's Open Ecosystem, said during a panel discussion that the responsibility of the cloud-native community is to bridge this gap. "You have to empathize with the customer, and the customer is the data scientist. A new cloud-native AI paper addresses those challenges, he said.
Cloud-native AI
Lachlan Evenson, principal product manager at Microsoft, said in the same panel as Gupta that a new role in the Kubernetes community also includes AI engineers, who sit between data scientists and infrastructure engineers or platform engineers.
During the panel discussion, Evenson noted that AI engineers need to understand not only all the terminology of the AI world, but also how to use these distributed systems at scale and build these new platforms.
The K8s promise: Scalable and secure
The founders of Kubernetes designed Kubernete to be stateless and later built stateful technology to integrate with its distributed platform.
"It's not just about this community, it's directly contributing to the scalability we've built on the platform through the K8s community," Evenson said. ”
"We need to provide open source alternatives and open source platforms so that companies that want to start investing and understand how AI is impacting their business, can adopt models without having to worry about data governance or security and start modifying and familiarizing them in their local environment. ”
Reference Links:
HTTPS://THENYSTOCK.IO/KUBERNETHES-AND-I-R-THE-A-FIT/