Kubernetes Gpu Management Just Got A Major Upgrade

“As a low-level systems engineer, if you do your occupation right, nary 1 knows you beryllium — but nan infinitesimal you do your occupation wrong, everybody knows you exist.”

That study from Nvidia Distinguished Engineer Kevin Klues underlines why nan Kubernetes unfastened root organization has been softly building foundational features and abstractions that will style really organizations tally AI workloads for nan adjacent decade.

At KubeCon + CloudNativeCon North America 2025 successful Atlanta, New Stack Founder and Publisher Alex Williams led a sheet chat pinch Klues and Jesse Butler, main merchandise head astatine Amazon Web Services, astir 2 developments that merit much attention: move assets allocation (DRA) and an upcoming workload abstraction that could toggle shape multinode AI deployments.

DRA: GPUs That Work Like Storage

Dynamic assets allocation (DRA), which reached wide readiness successful Kubernetes 1.34, solves nan long-standing vexation astir requesting GPU resources successful Kubernetes.

“The only knob you had successful nan aged measurement of requesting entree to resources was a elemental count,” Klues said. “You could say, ‘I want 2 GPUs,’ but you couldn’t opportunity what type of GPU. You couldn’t opportunity really you mightiness want that GPU to beryllium configured erstwhile it’s fixed to you.”

DRA, which Butler called “one of nan astir elegant things I’ve ever seen,” borrows its conceptual exemplary from persistent volumes and persistent measurement claims — acquainted abstractions that retention teams person utilized for years. The quality is that DRA useful pinch immoderate specialized hardware, not conscionable storage, meaning that third-party vendors tin now bring their ain instrumentality drivers and make hardware accessible to Kubernetes users successful standardized ways.

A New Workload Abstraction for Smart Scheduling

But DRA unsocial isn’t capable for analyzable AI deployments. Sometimes you request aggregate pods crossed aggregate nodes to each travel online simultaneously or, conversely, not astatine all. That’s nan problem a caller Kubernetes abstraction (called, simply, “the workload abstraction”) intends to solve.

“You want to beryllium capable to definitive things like, I tin person immoderate subset of these pods travel up, but if I can’t get each of them, I don’t want immoderate of them to travel up,” Klues said. “And, astatine slightest today, you can’t really definitive that successful nan Kubernetes world.”

A basal implementation is slated for nan Kubernetes 1.35  release connected Dec. 17, though Klues emphasized there’s important activity ahead. The abstraction will fto users specify pod groupings pinch scheduling constraints and topology requirements, benignant of for illustration node selectors connected steroids.

“It’s going to style nan early of really each of this useful for nan adjacent 10 years of Kubernetes,” Klues said, stressing that nan Device Management Working Group, wherever these features return shape, powerfully invites organization participation.

For nan afloat speech — including chat of agentic AI architectures, mini connection models, and why Unix accuracy still matters successful nan property of ample connection models — cheque retired nan complete interview.

Group Created pinch Sketch.