Part 4: Driver Implementation — How Developers Adapt to GPU-P

After understanding the architecture and isolation mechanisms of GPU-P, the most pressing question for a driver developer is: How do I implement and enable these features in my driver? This chapter will analyze the GPU-P development and adaptation process in detail from three dimensions: capability declaration, configuration files, and key DDIs (Device Driver Interfaces).

1. Capability Declaration: Telling the System "I'm Ready"

To let Windows know that your driver supports GPU partitioning, you first need to set specific capability bits in the host driver.

DXGK_VIDMMCAPS

When the KMD responds to the DxgkDdiQueryAdapterInfo call, it must populate the DXGK_DRIVERCAPS structure. Within it, MemoryManagementCaps.ParavirtualizationSupported is the "master switch" for enabling GPU-P:

// In the host KMD implementation
pDriverCaps->MemoryManagementCaps.ParavirtualizationSupported = TRUE;

Note: The situation is slightly different for MCDM (Microsoft Compute Driver Model) drivers. The host driver should set this bit to TRUE, while the guest driver running inside the virtual machine should adjust it based on whether it is in a virtualized environment.

2. INF File Configuration: The "Mover" of Resources

Since there is no KMD inside the virtual machine, the DLL images and registry settings required by the guest-side UMD must be provided in advance by the host.

Driver Store Mapping

In the INF file, you need to declare which files should be copied to the virtual machine. Commonly used registry entries include:

• CopyToVmOverwrite: Always overwrites the file with the same name in the virtual machine.
• CopyToVmWhenNewer: Only overwrites when the host file version is newer.

[DDInstall]
; Copies the host's driver files to the virtual machine's HostDriverStore directory
HKR,"CopyToVmOverwrite",SoftGpuFiles,%REG_MULTI_SZ%,"umd_binary.dll","umd_binary.dll"

These files are automatically placed in the virtual machine's %windir%\system32\HostDriverStore directory.

3. Key DDI Implementation: The Host's "Art of Management"

To manage numerous virtual machine instances, the host KMD needs to implement several key new DDIs:

DxgkDdiSetVirtualMachineData

This is the core interface through which Dxgkrnl communicates VM context to the KMD. When a new virtual machine instance starts or its configuration changes, the OS calls this interface to synchronize key metadata with the driver.

• Purpose: Passes virtual machine data, including the VM's unique identifier (LUID), video memory quota, compute resource limits, etc.
• Security Flag: Through the SecureVirtualMachine flag in DXGK_VIRTUALMACHINEDATAFLAGS, the OS tells the KMD whether the VM is in "secure mode" (e.g., Windows Sandbox).
• Developer Action: The driver should initialize or update its internal virtual machine tracking structures based on this data. For example, in secure mode, the driver must disable non-standard Escape calls and ensure that IOMMU isolation is fully effective. Furthermore, the driver can use this interface to associate the VM's guest space mapping with the host-side physical resources.

DxgkDdiQueryAdapterInfo (Extended)

This interface is one of the most functionally complex in WDDM. In a GPU-P environment, it is extended to support finer-grained cross-boundary queries.

• Key Flags:
  • VirtualMachineData: If this bit is TRUE, it indicates that the current query request originates from a virtual machine.
  • SecureVirtualMachine: Indicates whether the current execution is within a stricter security isolation environment (like a sandbox).
• Context Identification: The new hKmdProcessHandle member allows the driver, when processing queries from a virtual machine, to accurately identify and use the corresponding host-side process context.
• Common Query Types:
  • DXGKQAITYPE_GPUPCAPS: Queries the driver's partitioning capabilities, such as whether it supports Live Migration.
  • DXGKQAITYPE_GPUMMUCAPS: Queries the support status for GPU virtual addresses (GpuVA).
  • DXGKQAITYPE_PAGETABLELEVELDESC: Defines the page table hierarchy structure, which is crucial for the GpuMMU model.

DxgkDdiCreateProcess

In a virtualized environment, the host creates a "mirror process" object for each VM's drawing operations. The driver needs to recognize new flags in DXGK_CREATEPROCESSFLAGS to distinguish different execution contexts:

• VirtualMachineWorkerProcess: Corresponds to the host-side VM worker process (vmwp.exe). This process is responsible for managing the VM's virtual hardware (including vGPU emulation) but does not perform rendering itself. The driver can use this flag to skip the allocation of certain rendering resources.
• VirtualMachineProcess: Corresponds to the actual process inside the Guest that initiates drawing requests. Whenever an application within the virtual machine tries to use the GPU, a call is triggered on the host side via this flag.
• Process Association: Through the hKmdVmWorkerProcess handle, the driver can associate multiple guest-side processes with the same VM instance on the host, which is crucial for resource accounting, debugging, and performance monitoring (e.g., GPUView).
• Debugging Support: The pProcessName member provides a human-readable name for the process, greatly facilitating troubleshooting for developers in multi-VM concurrent scenarios.

4. Special Handling for MCDM Drivers

For compute cards without display output capabilities (like data center GPUs), adaptation to GPU-P must follow MCDM (Microsoft Compute Driver Model):

• Class Definition: The INF must specify Class=ComputeAccelerator.
• Capability Reduction: Display-related interfaces like DxgkDdiPresent do not need to be implemented, but GPU Virtual Address (GpuVA) and IOMMU isolation must be supported.
• Isolation Requirements: If an MCDM driver wants to run in a secure container, it must set IoMmuSecureModeSupported to TRUE.

5. Development Advice: The "Forbidden Zone" for Private Data

When writing a driver that supports GPU-P, developers must always keep the "cross-boundary" principle in mind:

1. Strictly Prohibited from Passing Pointers: The UMD is in the guest space, and the KMD is in the host space. Absolute physical memory addresses are invalid between the two.
2. Handle Translation: Utilize the DriverKnownEscape mechanism to let the OS translate resource handles from the guest side into handles recognizable on the host side.
3. Message Size Limit: The maximum size for a single VMBus message is typically 128KB. Avoid passing excessively large private data in a single AllocateCb call.

Conclusion

Adapting to GPU-P is not just about flipping a switch; it is a meticulous reconstruction of the driver architecture. Through standardized capability declarations and rigorous DDI implementation, developers can give their hardware a new lease on life in the cloud and virtualized environments.

In the next chapter, we will tackle the toughest nut to crack in the field of GPU virtualization: Live Migration.