Understanding GPU Virtual Addressing and Sparse Images/Buffers

Since the days of DirectX 11, and possibly even earlier, it's been possible to allocate memory on a GPU without actually using physical memory right away. But what does this mean?

Imagine you can create a buffer with a size of 64GB, even if your GPU only has 4GB of actual VRAM. How is this possible?

This works similarly to how virtual addresses work on a CPU. When you ask the operating system for memory, it doesn't immediately use real physical memory (RAM). Instead, it gives you a virtual address. The actual physical memory is only used when you start using that memory.

When you create a sparse buffer on a GPU, it only allocates a mapping table that looks something like this:

Page 0 = Address0
Page 1 = Address1
...

If you try to read this memory before it is backed by real memory, it will return zero because the memory doesn't actually exist yet.

Next, you allocate real, physical memory. This memory is usually aligned in pages (typically 64KB on modern GPUs). For example, let's say we allocate 2 pages, which equals 128KB. Then, we can bind these pages to the virtual address.

You can tell the GPU: "Bind my BufferAddress + 1GB (16384 pages) to the start of my allocated data." The mapping table then updates like this:

Page 16383 = NULL [previous value]
Page 16384 = AllocatedData + 0
Page 16385 = AllocatedData + 65536 Bytes
Page 16386 = NULL [previous value]

After binding the real memory to the virtual address, you can read or write to it in your shaders, compute passes, etc. Essentially, your 64GB buffer only takes up the size of the mapping table plus the 128KB of allocated real memory.