Intel 5-level paging

Short description: Processor extension for the x86-64 line of processors

A diagram of five levels of paging

Intel 5-level paging, referred to simply as 5-level paging in Intel documents, is a processor extension for the x86-64 line of processors. It extends the size of virtual addresses from 48 bits to 57 bits, increasing the addressable virtual memory from 256 TB to 128 PB. The extension was first implemented in the Ice Lake processors, and the 4.14 Linux kernel adds support for it. Windows 10 and 11 with server versions also support this extension in their latest updates, where it is provided by a separate kernel of the system called ntkrla57.exe.

Technology

x86-64 processors without this feature use a four-level page table structure when operating in 64-bit mode. A similar situation arose when the 32 bit IA-32 processors used two levels, allowing up to four GB of memory (both virtual and physical). To support more than 4 GB of RAM, an additional mode of address translation called Physical Address Extension (PAE) was defined, involving a third level. This was enabled by setting a bit in the CR4 register. Likewise, the new extension is enabled by setting bit 12 of the CR4 register (known as LA57). This is only used when the processor is operating in 64 bit mode, and only may be modified when it is not. If the bit is not set, the processor operates with four paging levels.

As adding another page table multiplies the address space by 512, the virtual limit has increased from 256 TB to 128 PB. An extra nine bits of the virtual address index the new table, so while formerly bits 0 through 47 were in use, now bits 0 through 56 are in use.

As with four level paging, the high-order bits of a virtual address that do not participate in address translation must be the same as the most significant implemented bit. With five-level paging enabled, this means that bits 57 through 63 must be copies of bit 56. Intel has renamed the existing paging system as "4-level paging", which used to be known as IA-32e paging.

Extending page table entry to 128 bits allows full 64-bit address space in 4-level paging scheme, and makes possible arbitrary allocation unit sizes in 5-level scheme.

Implementation

5-level paging is implemented by the Ice Lake microarchitecture, EPYC 9004 Series Processors and Storm peak Ryzen Threadripper PRO 7900WX series.

Support for the extension was submitted as a set of patches to the Linux kernel on 8 December 2016. As was reported on the Linux kernel mailing list, it consisted of extending the Linux memory model to use five levels rather than four. This is because, although Linux abstracts the details of the page tables, it still depends on having a number of levels in its own representation. When an architecture supports fewer levels, Linux emulates extra levels that do nothing. A similar change was previously made to extend from three levels to four.

Drawbacks

Adding another level of indirection makes page table "walks" longer. A page table walk occurs when either the processor's memory management unit or the memory management code in the operating system navigates the tree of page tables to find the page table entry corresponding to a virtual address. This means that, in the worst case, the processor or the memory manager has to access physical memory six times for a single virtual memory access, rather than five for the previous iteration of x86-64 processors. This results in slightly reduced memory access speed. In practice this cost is greatly mitigated by caches such as the translation lookaside buffer (TLB). Further extensions may reduce page walks by using 4096 128-bit page table entries, and allow a larger 64 KB page size and backward compatibility with 4 KB page operations.