tencent cloud

TencentOS Server

문서TencentOS ServerUser GuideSystem ConfigurationHugepages Management

Hugepages Management

다운로드
포커스 모드
폰트 크기
마지막 업데이트 시간: 2026-10-19 17:02:25
AI 번역
Hugepages is a memory allocation mechanism provided by the Linux kernel that allows the kernel to allocate memory in larger page sizes instead of the default small pages such as 4KB. This mechanism is particularly suitable for applications that require large amounts of contiguous memory space, such as databases, virtual machines, and certain high-performance computing tasks. Applications using hugepages can access large blocks of memory more directly without requiring the kernel to perform multiple page mappings, thereby improving memory access efficiency.
Although Linux provides interfaces such as proc for users to configure hugepages, these methods often act directly on the system rather than on specific services. The hugetlb parameter introduced by glibc adjusts malloc behavior at the process level, enabling it to request memory from hugepages in the way users expect, thereby improving performance without affecting global system settings.

Environment Variable Values

The configuration methods and values of the hugetlb environment variable are as follows:
Value
Description
Configuration Command
0
Do not use huge pages
export GLIBC_TUNABLES=glibc.malloc.hugetlb=0
1
Use Transparent Huge Pages (THP).
export GLIBC_TUNABLES=glibc.malloc.hugetlb=1
2
Use system default huge pages.
export GLIBC_TUNABLES=glibc.malloc.hugetlb=2
x(x > 2)
Specify the page size as x (must match a system-supported page size).
export GLIBC_TUNABLES=glibc.malloc.hugetlb=x

Using Transparent Hugepages

Checking the Enablement Status of Transparent Hugepages

cat /sys/kernel/mm/transparent_hugepage/enabled
Output example (square brackets indicate the current mode):
always [madvise] never
The meanings of the three modes are as follows:
Mode
Description
always
Always use transparent huge pages.
madvise
Use transparent huge pages based on application recommendations.
never
Never use transparent huge pages.
If always is set, the system-wide policy prioritizes transparent hugepages, which means this feature is enabled by default. If you can accept this setting, you can skip this section. If never is set, use the following command to adjust it:
echo madvise > /sys/kernel/mm/transparent_hugepage/enabled

Setting the glibc hugetlb Environment Variable

export GLIBC_TUNABLES=glibc.malloc.hugetlb=1

Test and verify

Compile and run the test program as described in the malloc & memset test case section. After the assignment is complete, check the AnonHugePages field in /proc/self/smaps. A non-zero value indicates that the process has used transparent hugepages, which means transparent hugepages have taken effect.

Using Standard Hugepages

When the hugetlb value is set to a number greater than 1, glibc supports two usage scenarios: using the system default hugepages, and allowing users to customize hugepages. These scenarios are described separately below.

System Default Hugepages

Checking the Size of System Default Hugepages

In fact, glibc also obtains this value by reading this field:
cat /proc/meminfo | grep Hugepagesize
Output example:
Hugepagesize:       2048 kB

Setting nr_hugepages

When using standard hugepages, you need to specify the number of pre-allocated hugepages first. As described in the previous section, the default hugepage size is 2 MB, so you need to set nr_hugepages corresponding to hugepages-2048kB:
ls /sys/kernel/mm/hugepages
Output example:
hugepages-1048576kB  hugepages-2048kB
echo 520 > /sys/kernel/mm/hugepages/hugepages-2048kB

Setting the glibc hugetlb Environment Variable

export GLIBC_TUNABLES=glibc.malloc.hugetlb=2

Test and verify

Compile and run the test program as described in the malloc & memset test case section. After the assignment is complete, check the fields in /proc/self/smaps where Private_Hugetlb is non-zero. You will get logs similar to the following:
MMUPageSize:           2048 kB
Private_Hugetlb:       1050624 kB
...
MMUPageSize:           2048 kB
Private_Hugetlb:       2048 kB
Here, MMUPageSize (Memory Management Unit Page Size) is 2048 kB (2 MB), which is consistent with the setting described in the Setting nr_hugepages section. Private_Hugetlb has two blocks: 1050624 kB and 2048 kB:
The first 1050624kB block is generated because malloc manages memory through chunks. In addition to providing 1 GB of memory, malloc also needs to reserve a chunk header to manage this memory block. Because 2 MB alignment is used in this case, this extra portion is generated.
The second 2048kB block is generated because malloc allocates space for the tcache data structure on its first call. Additionally, due to 2 MB alignment, this extra portion is produced.
Therefore, 520 hugepages of 2 MB each (1040 MB in total) are reserved here to verify the 1 GB scenario for malloc & memset. Reserving only 1 GB is insufficient because, in addition to the 1 GB memory itself, extra 2 MB-aligned space must be reserved for the chunk header and tcache.

Custom Hugepages

Checking Supported Hugepage Specifications

For example, the current environment supports two hugepage sizes: 1 GB and 2 MB. The default 2 MB size has already been verified, so this section uses 1 GB hugepages:
ls /sys/kernel/mm/hugepages
Output example:
hugepages-1048576kB  hugepages-2048kB

Setting nr_hugepages

As in the previous section, you need to reserve hugepages:
echo 3 > /sys/kernel/mm/hugepages/hugepages-1048576kB
Three 1 GB hugepages are reserved here to verify the 1 GB scenario for malloc & memset.

Setting the glibc hugetlb Environment Variable

Warning: Note that the unit here is B (bytes), so manual conversion is required.
export GLIBC_TUNABLES=glibc.malloc.hugetlb=1073741824

Test and verify

Compile and run the test program as described in the malloc & memset test case section. After the assignment is complete, check the fields in /proc/self/smaps where Private_Hugetlb is non-zero. You will get logs similar to the following:
MMUPageSize:           1048576 kB
Private_Hugetlb:       2097152 kB
...
MMUPageSize:           1048576 kB
Private_Hugetlb:       1048576 kB
Here, MMUPageSize (Memory Management Unit Page Size) is 1048576 kB (1 GB), which is consistent with the setting described in the Setting nr_hugepages section. Private_Hugetlb has two blocks: 2097152 kB and 1048576 kB:
The first 2097152kB block is still generated because space is reserved for the chunk data structure plus page alignment.
The second 1048576kB block is still generated because of tcache.
It should be noted that the extra 1048576 kB is not left unused. If small memory allocations occur later, malloc still prioritizes carving memory from this space. However, the above example also reveals a drawback of hugepages: improper use can cause significant memory waste, so developers need to plan memory usage carefully.

malloc & memset Test Cases

The following test program can be used to verify whether transparent hugepages and standard hugepages have taken effect. The program allocates and initializes 1 GB of memory, and then determines hugepage usage by checking the /proc/<PID>/smaps output of the process.

Test Program

Create the test program test_hugepage.c:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

#define MEM_SIZE (1UL << 30)  /* 1GB */

int main()
{
    char *p = malloc(MEM_SIZE);
    if (!p) {
        perror("malloc failed");
        return 1;
    }

    printf("PID: %d\\n", getpid());

    /* Initialize memory to trigger hugepage allocation */
    memset(p, 0, MEM_SIZE);

    printf("Allocated and initialized %lu MB of memory. Press Enter to exit...\\n", MEM_SIZE >> 20);
    getchar();

    free(p);
    return 0;
}

Compiling and Running

gcc -o test_hugepage test_hugepage.c
./test_hugepage
After the program starts, it prints the process PID and waits for input. At this point, run the following command in another terminal to check hugepage usage:
cat /proc/<PID>/smaps | grep -E "AnonHugePages|Private_Hugetlb|MMUPageSize"

Verification Methods

Based on the configured hugepage type, check whether the corresponding field in smaps is non-zero:
Huge Page Type
smaps Field
Non-Zero Meaning
Transparent Huge Pages (hugetlb=1)
AnonHugePages
The process has used transparent huge pages.
Standard Huge Pages (hugetlb=2 or a custom value)
Private_Hugetlb
The process has used standard huge pages.
Note:
For transparent hugepages, a non-zero AnonHugePages value indicates that transparent hugepages have taken effect. For standard hugepages, a non-zero Private_Hugetlb value and an MMUPageSize that matches the configured hugepage size indicate that standard hugepages have taken effect.

How It Works

The principles involved in this feature are as follows:
0f982c1827  malloc: Enable huge page support on main arena
0849eed45d  malloc: Move MORECORE fallback mmap to sysmalloc_mmap_fallback
c1beb51d08  malloc: Add Huge Page support to arenas
98d5fcb8d0  malloc: Add Huge Page support for mmap
6cc3ccc67e  malloc: Move mmap logic to its own function
7478c9959a  malloc: Add THP/madvise support for sbrk
5f6d8d97c6  malloc: Add madvise support for Transparent Huge Pages

Variable Resolution Process

On the first malloc call, a function that parses the glibc hugetlb environment variable is invoked. The code is as follows, where value is the user-specified value:
static __always_inline int
do_set_hugetlb (size_t value)
{
  if (value == 1)
    {
      enum malloc_thp_mode_t thp_mode = __malloc_thp_mode ();
      /* Only enable THP madvise usage if system does support it and
         has 'madvise' mode.  Otherwise the madvise() call is wasteful.  */
      if (thp_mode == malloc_thp_mode_madvise)
        mp_.thp_pagesize = __malloc_default_thp_pagesize ();
    }
  else if (value >= 2)
    __malloc_hugepage_config (value == 2 ? 0 : value,
                              &mp_.hp_pagesize, &mp_.hp_flags);
  return 0;
}
value = 1 (transparent hugepages): When value is 1, the thp_pagesize value is set by reading /sys/kernel/mm/transparent_hugepage/hpage_pmd_size through the __malloc_default_thp_pagesize function:
unsigned long int
__malloc_default_thp_pagesize (void)
{
  int fd = __open64_nocancel (
    "/sys/kernel/mm/transparent_hugepage/hpage_pmd_size", O_RDONLY);
  if (fd == -1)
    return 0;

  char str[INT_BUFSIZE_BOUND (unsigned long int)];
  ssize_t s = __read_nocancel (fd, str, sizeof (str));
  __close_nocancel (fd);
  if (s == 0)
    return 0;

  unsigned long int r = 0;
  for (ssize_t i = 0; i < s; i++)
    {
      if (str[i] == '\\n')
        break;
      r *= 10;
      r += str[i] - '0';
    }
  return r;
}
value = 2 (system default hugepages): When value is 2, the system default hugepage size is obtained through malloc_default_hugepage_size.
value > 2 (custom hugepages): When value is greater than 2, read /sys/kernel/mm/hugepage to determine whether the input value is a hugepage size supported by the system:
void
__malloc_hugepage_config (size_t requested, size_t *pagesize, int *flags)
{
  *pagesize = 0;
  *flags = 0;

  if (requested == 0)
    {
      *pagesize = malloc_default_hugepage_size ();
      if (*pagesize != 0)
        *flags = hugepage_flags (*pagesize);
      return;
    }

  /* Each entry represents a supported huge page in the form of:
     hugepages-<size>kB.  */
  int dirfd = __open64_nocancel ("/sys/kernel/mm/hugepages",
                                 O_RDONLY | O_DIRECTORY, 0);
  if (dirfd == -1)
    return;

  char buffer[1024];
  while (true)
    {
      ...
      hpsize *= 1024;

      if (hpsize == requested)
        {
          *pagesize = hpsize;
          *flags = hugepage_flags (*pagesize);
          __close_nocancel (dirfd);
          return;
        }
    }

  __close_nocancel (dirfd);
}
The implementation of hugepage_flags is as follows. flags is ultimately passed to mmap to specify the size of standard hugepages:
static inline int
hugepage_flags (size_t pagesize)
{
  return MAP_HUGETLB | (__builtin_ctzll (pagesize) << MAP_HUGE_SHIFT);
}

madvise Encapsulation

For scenarios that use transparent hugepages, when /sys/kernel/mm/transparent_hugepage/enabled is set to madvise, user space must use the madvise system call to tell the kernel the usage pattern of the specified virtual memory. glibc wraps madvise as follows:
static inline void
madvise_thp (void *p, INTERNAL_SIZE_T size)
{
# ifdef MADV_HUGEPAGE
  /* Do not consider areas smaller than a huge page or if the tunable
     is not active.  */
  if (mp_.thp_pagesize == 0 || size < mp_.thp_pagesize)
    return;

  ...

  __madvise (p, size, MADV_HUGEPAGE);
# endif
}
thp_pagesize is initialized to the system default transparent hugepage size on the first malloc call. glibc can use transparent hugepages by calling madvise_thp after each mmap.

Summary

Hugepages can improve memory access efficiency and performance, but they can also cause issues such as memory waste. The glibc hugetlb environment variable works at a finer granularity to avoid excessive memory consumption by hugepages while retaining their efficient memory usage and ensuring proper memory planning.
이전 주제: cgroup Resource Limits다음 주제: Software Package Management

도움말 및 지원

문제 해결에 도움이 되었나요?

피드백

키워드로 검색
    Enter keywords to search documentation