How do I configure HugePages on my Amazon EC2 Linux instance?

Last updated: 2020-06-15

I want to improve performance on my database or application hosted on an Amazon Elastic Compute Cloud (Amazon EC2) Linux instance. How can I configure HugePages on my instance to improve performance?

Short description

HugePages are memory pages that are larger in size than the default 4 KB page size. Enabling HugePages in your Linux instance’s kernel in a large application or database environment helps reduce the stress on the TLB. Less stress on the TLB increases your server and application or database performance because TLB stores fewer pages that are larger in size.

For example, if TLB entries are at 512 without HugePages configured on the instance, then the memory size is approximately equal to the following:

4096 B * 512 = 2 MB memory size

If TLB entries are at 512 with HugePages configured on the instance, then the memory size is approximately equal to the following:

2 MB * 512 = 1 GB memory size

Note: Valid values for HugePages in a Linux system are 2 MB and 1 GB. The default value is 2 MB.

For more information about HugePages, see HugePages on the Oracle Help Center.

Keep the following points in mind before enabling HugePages:

  • By default, starting your system doesn't release HugePages. To enable release after system startup, modify the number of pages available or modify the HugePages pool size.
  • If the appropriate space for allocation isn't available in memory during system startup, then HugePages allocation fails. The value of hugepagesize * number of HugePages must always be less than the total memory of the instance. To determine the amount memory that must be available verify the total memory of the instance by running the free -m command.
    Note: Get the hugepagesize value from your application or database vendor or from your system administrator. For example, to enable HugePages for an Oracle database, you can get the SGA from Oracle. For more information, see Viewing SGA memory allocation in the Database Performance Tuning Guide.
  • Enabling Hugepages in the instance's kernel requires that the application or database on the instance supports HugePages. Verify with your vendor or your system administrator that your application or database supports HugePages.

Resolution

Note: It's a best practice to configure HugePages in a test environment and benchmark performance before deploying it to production.

1.    To verify that your kernel doesn't currently have HugePages enabled, run the cat command:

# sudo cat /proc/sys/vm/nr_hugepages
0

In the preceding example, the nr_hugepages parameter value is 0, indicating that HugePages isn't enabled.

2.    To enable HugePages, run the following command to set the kernel parameter value to 2048:

# sudo sysctl -w vm.nr_hugepages=2048
vm.nr_hugepages = 2048

3.    To verify that the system allocates HugePages after reboot, add the following entry to /etc/sysctl.conf:

# sudo echo "vm.nr_hugepages=2048" >> /etc/sysctl.conf

4.    Run the cat command to verify the change:

# sudo cat /etc/sysctl.conf
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
vm.zone_reclaim_mode=1
vm.nr_hugepages=2048

5.    Reboot your instance.

6.    After rebooting, run the cat command again to verify the HugePages value and configuration:

# sudo cat /proc/sys/vm/nr_hugepages
2048

7.    Check the available HugePages in /proc/meminfo:

# sudo grep Huge /proc/meminfo
HugePages_Total:    2048
HugePages_Free:     2048
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

For definitions of each variable in the preceding example output, see HugePages on kernel.org.

HugePages is now enabled on your server.

8.    Configure HugePages on additional applications as needed.

Note: To disable HugePages:

1.    Change the nr_hugepages back to 0.

2.    Remove the entry in sysctl.conf.

3.    Reboot the server to take effect.


Did this article help you?

Anything we could improve?


Need more help?