How To Get Bare-metal Gpu Performance In Confidential Vms

PARIS — At OpenInfra Summit Europe 2025, NVIDIA wanted to make it very clear to AI developers, operators and users: If you want to tally sensitive AI workloads connected GPUs anyplace — connected premises, successful nationalist clouds aliases at nan edge — you request some virtual instrumentality (VM)-level sandboxing and hardware-backed representation confidentiality. That means, said Zvonko Kaiser, NVIDIA main systems engineer, you should harvester Kata Containers (lightweight VMs for containers) pinch confidential computing to sphere bare-metal GPU capacity while preventing nan unreality usability from inspecting your exemplary and data.

Kata, for those of you who don’t know, is an unfastened root task that combines lightweight VMs pinch instrumentality runtimes. It uses hardware virtualization exertion to motorboat a abstracted VM for each container, providing beardown isolation betwixt containers. Each container, successful turn, runs a minimal, stripped-down Linux kernel. Kata Containers purpose to connection nan capacity benefits of containers on pinch nan information and workload isolation of VMs.

Understanding Kata Containers and Lightweight VMs

“Kata is nan micro-VM … it conscionable fits into nan unreality autochthonal space,” Kaiser told nan audience. He based on that Kata gives nan isolation instrumentality runtimes deficiency while still integrating pinch Kubernetes workflows.

What confidential computing brings to nan array is in-memory information and exertion encryption. We’ve agelong had information by encryption erstwhile information is astatine remainder aliases successful transit connected nan network. Now, we person it successful representation arsenic well.

The constituent of combining them, Kaiser explained, is simply a flip of nan accepted threat model. Classic Kata usage assumes nan workload is untrusted, truthful it protects nan big from nan container. Confidential computing, utilizing CPU information features specified arsenic SEV/TDX, holds that: “We do not spot nan infrastructure.” Thus, by encrypting nan VM, moreover your unreality supplier cannot snapshot aliases inspect impermanent memory.

The Role of Confidential Computing and Attestation

To make judge this really works, he emphasized nan value of attestation arsenic nan system that glues nan stack together. Only aft a cryptographic impervious that nan VM and its boot/guest authorities lucifer an expected configuration should secrets aliases keys beryllium released to a workload. This enables a full-stack spot exemplary crossed nan power plane, worker nodes and pods. “The process of proving that your authorities … is really nan authorities that you are measuring” is halfway to confidential deployments, said Kaiser.

Where AI and NVIDIA travel together is by utilizing these to alteration you to usage GPUs for illustration bare metallic wrong confidential VMs. Kaiser explained really NVIDIA is moving to make GPU workloads “lift-and-shift” into Kata/confidential VMs without losing capacity aliases functionality.

Achieving Bare-Metal GPU Performance for AI Workloads

To do this, NVIDIA leverages Kubernetes building blocks, nan GPU Operator and Container Device Interface (CDI) — truthful that drivers, libraries and instrumentality mappings are presented to containers precisely arsenic they would beryllium connected bare metal. “We conscionable took this shape that we person already connected bare metallic and conscionable put it into nan extremity truthful that nan instrumentality that’s moving successful Kata will consciousness and behave nan very aforesaid arsenic moving connected bare metal.”

That effort includes support for PCIe pass-through, Single Root IO Virtualization (SR-IOV), GPUDirect Remote Direct Memory Access (RDMA) and per-pod runtime configurations truthful 1 pod tin usage PF pass-through while different uses SR-IOV. Crucially, Kata’s reliance connected nan impermanent kernel decouples personification abstraction from big kernel changes. This reduces nan consequence that a big update will break GPU drivers wrong nan workload VM.

Solving PCIe Topology Challenges With NVIDIA’s VRA

That whitethorn sound complex, but, according to Kaiser, nan existent difficult portion is nan topology. NVIDIA’s reply is its Virtualization Reference Architecture (VRA). NVIDIA will soon beryllium publishing successful much item this attack of addressing nan thorny problem of PCIe topology and peer-to-peer GPU connection wrong VMs. It supports 2 approaches:

• Flatten nan hierarchy: In this approach, you simplify topology to make provisioning easier. Cloud providers are already sometimes utilizing this for confidential AI deployments, but it comes astatine nan costs of hiding useful peer-to-peer links.
• Host-topology replication: Detect nan host’s PCIe/input–output representation guidance portion (IOMMU) layout and reflector it wrong nan guest, preserving PCIe Address Translation Services (ATS) and PCIe Access Control Services (ACS) flags, which enables GPU peer-to-peer DMA and GPUDirect behavior.

Why two? So “You tin either flatten nan level because you opportunity you don’t attraction astir nan level … aliases you tin opportunity ‘I want big replication because I’m doing P2P objects.’ So some modes are supported,” Kaiser explained.

NVIDIA besides explained applicable workarounds for IOMMU grouping and PCIe slot limits. For example, you tin selectively representation only required GPU devices to impermanent guidelines ports while leaving unrelated peripherals connected span ports. This avoids unnecessary instrumentality pass-through and complexity.

Kaiser said NVIDIA is collaborating pinch Red Hat, IBM and nan unfastened root Kata organization to upstream nan VRA and tooling, including host-topology discovery and capacity guides. Other upcoming publications covered CPU pinning, ACS/ATS settings, and GPUDirect/RDMA tuning for confidential VMs, and emphasized avoiding nested virtualization truthful operators tin tally VM arsenic a Service patterns astatine L1 pinch accordant attestation crossed layers. In short, “We want to upstream everything truthful that group tin replicate it arsenic a reference architecture,” said Kaiser.

Open Source Collaboration and Upstreaming Efforts

All that sounds great, but Kaiser was observant to statement trade-offs. Combining Kata pinch confidential computing is not a metallic bullet. VM breakouts stay a theoretical risk; confidential VMs trim a provider’s expertise to inspect representation but do not destruct each onslaught surfaces. Still, nan mixed attack substantially reduces nan opportunity for unreality operators aliases co-tenants to entree delicate exemplary artifacts aliases training data.

Still, erstwhile published and available, NVIDIA’s attack to moving delicate AI workloads astatine standard will almost surely lead to a caller AI stack that combines lightweight VM isolation (Kata), hardware representation encryption and attestation (confidential computing) and GPU instrumentality mapping abstractions (CDI + GPU Operator) pinch observant handling of PCIe topology and IOMMU constraints to sphere information and performance.

Group Created pinch Sketch.