Skip to content

Add 64K page size support for ARM64#12656

Open
nblintao wants to merge 2 commits intogoogle:masterfrom
nblintao:arm64-64k-pagesize
Open

Add 64K page size support for ARM64#12656
nblintao wants to merge 2 commits intogoogle:masterfrom
nblintao:arm64-64k-pagesize

Conversation

@nblintao
Copy link

@nblintao nblintao commented Feb 26, 2026

This adds support for running gVisor on ARM64 systems with 64K page size (e.g., NVIDIA Grace CPU).

Key changes:

  1. hostarch: Split page size constants into separate files selected by the pagesize_64k build tag. 64K pages use PageShift=16 and HugePageShift=29 (512MB huge pages). Remove uint16 from the bytecount interface since 64K (2^16) cannot be represented by uint16.

  2. ring0: Move page table layout constants to page-size-specific files. 64K pages use larger page table entries (64K per table) and skip the PUD level.

  3. ring0/pagetables: Add 3-level page table walker for 64K pages (PGD→PMD→PTE, no PUD). Each level indexes 8192 entries (13 bits) with 512MB sect pages at the PMD level.

  4. sentry/arch: Fix mmap layout constants that overflowed the 48-bit address space with 64K pages. Use fixed values for maxMmapRand64 (1<<45 = 32TB) and minMmapRand64 (1<<30 = 1GB), and compute maxStackRand64 using hostarch.PageShift.

  5. Bazel build system: Add --define=pagesize=64k support that propagates the pagesize_64k Go build tag to all binaries and tests.

Build with bazel build --define=pagesize=64k //runsc:runsc to enable.

Note: Currently only the ptrace platform is supported for 64K pages. KVM and systrap platforms require additional work.

Updates #8196

@tranji-cloud tranji-cloud self-assigned this Feb 26, 2026
@@ -0,0 +1,37 @@
// Copyright 2019 The gVisor Authors.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Update copyright to 2026 for the brand new files

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

// PageSize is the system page size.
// arm64 support 4K/16K/64K page size,
// which can be get by unix.Getpagesize().
// Currently, only 4K page size is supported.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, only 4K page size is supported by default, or 64k pages by using the pagesize_64k build tag.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

@konstantin-s-bogom
Copy link
Member

I think it's too early to add the "Fixes #8196" tag since both KVM and Systrap don't work.

In line with this, can you add checks against expected hostarch.PageSize in the New() constructors for both KVM and Systrap so that they fail to start.

Otherwise, generally LGTM.

@nblintao
Copy link
Author

I think it's too early to add the "Fixes #8196" tag since both KVM and Systrap don't work.

In line with this, can you add checks against expected hostarch.PageSize in the New() constructors for both KVM and Systrap so that they fail to start.

Fixed.

This adds support for running gVisor on ARM64 systems with 64K page
size (e.g., NVIDIA Grace CPU).

Key changes:

1. hostarch: Split page size constants into separate files selected by
   the `pagesize_64k` build tag. 64K pages use PageShift=16 and
   HugePageShift=29 (512MB huge pages). Remove uint16 from the
   bytecount interface since 64K (2^16) cannot be represented by uint16.

2. ring0: Move page table layout constants to page-size-specific files.
   64K pages use larger page table entries (64K per table) and skip the
   PUD level.

3. ring0/pagetables: Add 3-level page table walker for 64K pages
   (PGD→PMD→PTE, no PUD). Each level indexes 8192 entries (13 bits)
   with 512MB sect pages at the PMD level.

4. sentry/arch: Fix mmap layout constants that overflowed the 48-bit
   address space with 64K pages. Use fixed values for maxMmapRand64
   (1<<45 = 32TB) and minMmapRand64 (1<<30 = 1GB), and compute
   maxStackRand64 using hostarch.PageShift.

5. Bazel build system: Add `--define=pagesize=64k` support that
   propagates the `pagesize_64k` Go build tag to all binaries and tests.

Build with `bazel build --define=pagesize=64k //runsc:runsc` to enable.

Note: Currently only the ptrace platform is supported for 64K pages.
KVM and systrap platforms require additional work.

Updates google#8196
- Update copyright year to 2026 for all new files
- Update comment in hostarch_arm64.go to mention 64K page support
  via pagesize_64k build tag
- Add PageSize checks in KVM and Systrap New() constructors to
  fail early on non-4K page sizes
@konstantin-s-bogom
Copy link
Member

One more thing, can you give a test run of at least these targets on a 64k ARM instance: bazel query //test/syscalls/... | grep ptrace?

We don't have any CI runners configured for it currently.

@nblintao
Copy link
Author

One more thing, can you give a test run of at least these targets on a 64k ARM instance: bazel query //test/syscalls/... | grep ptrace?

We don't have any CI runners configured for it currently.

I ran the ptrace syscall tests on a 64K ARM64 machine (NVIDIA Grace CPU):

$ bazel query '//test/syscalls/...' | grep ptrace | wc -l
665

$ bazel test --define=pagesize=64k \
    --@io_bazel_rules_go//go/config:tags=pagesize_64k \
    --test_tag_filters=runsc_ptrace \
    --test_output=errors \
    //test/syscalls/...
...
Executed 654 out of 654 tests: 435 tests pass and 219 fail locally.

Failure breakdown:

  • 211 save/restore tests — all fail with failed to load kernel: header error: EOF during restore. Save/restore serialization hasn't been adapted for 64K pages yet.
  • 5 page-size-related failuresexec_binary_test (ELF segment layout), proc_pid_smaps_test (reports 4KB instead of 64KB), packet_mmap_test (ring buffer page alignment), splice_test and pipe_test (pipe capacity assumptions). These are in various subsystems that need 64K page adaptation in follow-up PRs.
  • 3 likely unrelatedstat_test, link_test, rename_test fail on link counts and errno differences, likely pre-existing ptrace platform issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants