Database Manual / Self-Managed Deployments / Administration / Performance

TCMalloc Performance Optimization for a Self-Managed Deployment用于自我管理部署的TCMalloc性能优化

Changed in version 8.0.在版本8.0中的更改。

Starting in MongoDB 8.0, MongoDB uses an upgraded version of TCMalloc that uses per-CPU caches, instead of per-thread caches, to reduce memory fragmentation and make your database more resilient to high-stress workloads.从MongoDB 8.0开始,MongoDB使用TCMalloc的升级版本,该版本使用每CPU缓存,而不是每线程缓存,以减少内存分片,使数据库更能适应高压力的工作负载。

To use the new TCMalloc version:要使用新的TCMalloc版本:

Platform Support平台支持

Operating systems that support MongoDB 8.0 also support the updated TCMalloc, except for the following:支持MongoDB 8.0的操作系统也支持更新的TCMalloc,但以下情况除外:

  • RHEL 8 / Oracle 8 on the PPC64LE and s390x architecturesPPC64LEs390x架构上的RHEL 8/Oracle 8
  • RHEL 9 / CentOS 9 / Oracle 9 on the PPC64LE architecture

These operating systems use the legacy TCMalloc version. If you use these operating systems, disable THP.这些操作系统使用传统的TCMalloc版本。如果您使用这些操作系统,请禁用THP

Windows uses the legacy TCMalloc version and does not support the updated TCMalloc.Windows使用旧版TCMalloc,不支持更新的TCMalloc。

Enable Transparent Hugepages启用透明护圈 (THP)

Transparent Hugepages (THP) is a Linux memory management system that reduces the overhead of Translation Lookaside Buffer (TLB) lookups. THP achieves this by combining small pages and making them appear as larger memory pages to the application.透明缓冲区(THP)是一个Linux内存管理系统,可以减少翻译查找缓冲区(TLB)查找的开销。THP通过组合小页面并使其在应用程序中显示为更大的内存页面来实现这一点。

In MongoDB 8.0 and later, ensure that THP is enabled before mongod starts by creating a service file for your platform's initialization system. If you are using MongoDB 7.0 or earlier, disable THP.在MongoDB 8.0及更高版本中,通过为平台的初始化系统创建服务文件,确保在mongod启动之前启用THP。如果您使用的是MongoDB 7.0或更早版本,请禁用THP。

Note

THP operates at a system level, so it applies to all processes under the system. You cannot enable or disable THP per-process level.THP在系统级别运行,因此它适用于系统下的所有流程。您无法按进程级别启用或禁用THP。

Additionally, for RHEL and CentOS systems that use ktune and tuned performance profiles, you must also create a custom tuned profile.此外,对于使用ktunetuned性能配置文件的RHEL和CentOS系统,您还必须创建自定义tuned配置文件。

Create a Service File创建服务文件

To create a service file that enables THP, use the built-in initialization system for your platform. 要创建启用THP的服务文件,请使用您平台的内置初始化系统。Recent versions of Linux typically use systemd, which uses the systemctl command. 最新版本的Linux通常使用systemd,它使用systemctl命令。Older versions of Linux tend to use System V init, which uses the service command. For more information, see the documentation for your operating system.旧版本的Linux倾向于使用System V init,它使用service命令。有关更多信息,请参阅操作系统的文档。

Use the initialization system for your platform:使用您平台的初始化系统:

systemd(systemctl)
1

Create the systemd unit file创建systemd单元文件

Create the following file and save it at /etc/systemd/system/enable-transparent-huge-pages.service:创建以下文件并将其保存在/etc/systemd/system/enable-transparent-huge-pages.service

[Unit]
Description=Enable Transparent Hugepages (THP)
DefaultDependencies=no
After=sysinit.target local-fs.target
Before=mongod.service

[Service]
Type=oneshot
ExecStart=/bin/sh -c 'echo always | tee /sys/kernel/mm/transparent_hugepage/enabled > /dev/null && echo defer+madvise | tee /sys/kernel/mm/transparent_hugepage/defrag > /dev/null && echo 0 | tee /sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_none > /dev/null && echo 1 | tee /proc/sys/vm/overcommit_memory > /dev/null'

[Install]
WantedBy=basic.target

Note

Some versions of Red Hat Enterprise Linux, and potentially other Red Hat-based derivatives, use a different path for the THP enabled file:Red Hat Enterprise Linux的某些版本,以及可能基于Red Hat的其他衍生版本,对启用THP的文件使用了不同的路径:

/sys/kernel/mm/redhat_transparent_hugepage/enabled

Verify which path is in use on your system and update the enable-transparent-huge-pages.service file accordingly.验证系统上正在使用哪个路径,并相应地更新enable-transparent-huge-pages.service文件。

2

Reload systemd unit files重新加载systemd单元文件

To reload the systemd unit files and make enable-transparent-huge-pages.service available for use, run the following command :要重新加载systemd单元文件并启用enable-transparent-huge-pages.service,请运行以下命令:

sudo systemctl daemon-reload
3

Start the service启动服务

Run:运行:

sudo systemctl start enable-transparent-huge-pages

To verify that the relevant THP settings have changed, run the following command:要验证相关THP设置是否已更改,请运行以下命令:

cat /sys/kernel/mm/transparent_hugepage/enabled && cat /sys/kernel/mm/transparent_hugepage/defrag && cat /sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_none && cat /proc/sys/vm/overcommit_memory

On Red Hat Enterprise Linux and potentially other Red Hat-based derivatives, you may instead need to use the following:在Red Hat Enterprise Linux和其他可能基于Red Hat的衍生产品上,您可能需要使用以下内容:

cat /sys/kernel/mm/redhat_transparent_hugepage/enabled && cat /sys/kernel/mm/redhat_transparent_hugepage/defrag && cat /sys/kernel/mm/redhat_transparent_hugepage/khugepaged/max_ptes_none && cat /proc/sys/vm/overcommit_memory

The output should resemble the following:输出应类似于以下内容:

always
defer+madvise
0
1
4

Configure your operating system to run it on boot.配置操作系统以在启动时运行它。

To ensure that this setting is applied each time the operating system starts, run the following command:为确保每次启动操作系统时都应用此设置,请运行以下命令:

sudo systemctl enable enable-transparent-huge-pages
5

(Optional) Customize tuned or ktune profile可选)自定义调谐或ktune配置文件

If you use tuned or ktune proffiles on RHEL/ CentOS, you must also create a custom tuned profile.

System V Init(service)
1

Create the init.d script创建init.d脚本

Create the following file and save it at /etc/init.d/enable-transparent-hugepages:创建以下文件并将其保存在/etc/init.d/enable-transparent-hugepages

#!/bin/bash
### BEGIN INIT INFO
# Provides: enable-transparent-hugepages
# Required-Start: $local_fs
# Required-Stop:
# X-Start-Before: mongod mongodb-mms-automation-agent
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Enable Linux Transparent Hugepages
# Description: Enable Linux Transparent Hugepages, to improve
# database performance.
### END INIT INFO

case $1 in
start)
if [ -d /sys/kernel/mm/transparent_hugepage ]; then
thp_path=/sys/kernel/mm/transparent_hugepage
elif [ -d /sys/kernel/mm/redhat_transparent_hugepage ]; then
thp_path=/sys/kernel/mm/redhat_transparent_hugepage
else
return 0
fi

echo 'always' | tee ${thp_path}/enabled > /dev/null && echo defer+madvise | tee ${thp_path}/defrag > /dev/null && echo 0 | tee ${thp_path}/khugepaged/max_ptes_none > /dev/null && echo 1 | tee /proc/sys/vm/overcommit_memory > /dev/null'

unset thp_path
;;
esac
2

Make the script executable使脚本可执行

Run:运行:

sudo chmod 755 /etc/init.d/enable-transparent-hugepages
3

Run the script运行脚本

Run:运行:

sudo /etc/init.d/enable-transparent-hugepages start

To verify that the relevant THP settings have changed, run the following command:要验证相关THP设置是否已更改,请运行以下命令:

cat /sys/kernel/mm/transparent_hugepage/enabled && cat /sys/kernel/mm/transparent_hugepage/defrag && cat /sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_none && cat /proc/sys/vm/overcommit_memory

On Red Hat Enterprise Linux and potentially other Red Hat-based derivatives, you may instead need to use the following:在Red Hat Enterprise Linux和其他可能基于Red Hat的衍生产品上,您可能需要使用以下内容:

cat /sys/kernel/mm/redhat_transparent_hugepage/enabled

The output should resemble the following:输出应类似于以下内容:

always
defer+madvise
0
1
4

Configure your operating system to run it on boot配置操作系统以在启动时运行它

To ensure that this setting is applied each time the operating sytem starts, run the following command for your Linux distribution:为确保每次启动操作系统时都应用此设置,请为Linux发行版运行以下命令:

DistributionCommand
Ubuntu and Debian
sudo update-rc.d enable-transparent-hugepages defaults
SUSE
sudo insserv /etc/init.d/enable-transparent-hugepages
Red Hat, CentOS, Amazon Linux, and derivatives
sudo chkconfig --add enable-transparent-hugepages
5

(Optional) Customize tuned or ktune profile自定义调谐或ktune配置文件

If you are using tuned or ktune profiles on RHEL/ CentOS, you must also create a custom tuned profile.

Using tuned and ktune

Important

If you use tuned or ktune, perform the steps in this section after creating the service file.如果使用tunedktune,请在创建服务文件后执行本节中的步骤。

tuned and ktune are kernel tuning utilities that can affect the Transparent Hugepages setting on your system. tunedktune是内核调优实用程序,可以影响系统上的透明拥抱设置。If you are using tuned or ktune on your RHEL or CentOS system while running mongod, you must create a custom tuned profile to ensure that THP stays enabled.如果在运行mongod时在RHEL或CentOS系统上使用tunedktune,则必须创建自定义tuned配置文件以确保THP保持启用状态。

1

Create a new profile创建新配置文件

Create a new directory to store the custom tuned profile. The following example inherits from the existing virtual-guest profile, and uses virtual-guest-thp as the new profile:创建一个新目录来存储自定义调优的配置文件。以下示例继承了现有的virtual-guest配置文件,并使用virtual-guest-thp作为新配置文件:

sudo mkdir /etc/tuned/virtual-guest-thp
2

Edit 编辑tuned.conf

Create and edit /etc/tuned/virtual-guest-thp/tuned.conf. Add the following text:创建和编辑/etc/tuned/virtual-guest-thp/tuned.conf。添加以下文本:

[main]
include=virtual-guest

[vm]
transparent_hugepages=always

This example inherits from the existing virtual-guest profile. Select the appropriate profile for your system.此示例继承自现有的virtual-guest配置文件。为系统选择合适的配置文件。

3

Enable the new profile启用新配置文件

Run:运行:

sudo tuned-adm profile virtual-guest-thp

Enable Per-CPU Caches启用每CPU缓存

To verify that TCMalloc is running with per-CPU caches, ensure that:要验证TCMalloc是否使用每CPU缓存运行,请确保:

If per-CPU caches aren't enabled, ensure that:如果未启用每CPU缓存,请确保:

  • You disable glibc rseq.您禁用glibc rseq。
  • You're using Linux kernel version 4.18 or later.您使用的是Linux内核版本4.18或更高版本。

Disable glibc rseq禁用glibc-rseq

The new TCMalloc requires Restartable Sequences (rseq) to implement per-CPU caches. 新的TCMalloc需要可重启序列(rseq)来实现每个CPU的缓存。If another application, such as the glibc library, registers an rseq structure before TCMalloc, TCMalloc can't use rseq. Without rseq, TCMalloc uses per-thread caches, which are used by the legacy TCMalloc version.如果另一个应用程序(如glibc库)在TCMalloc之前注册了rseq结构,则TCMalloc不能使用rseq。如果没有rseq,TCMalloc将使用传统TCMalloc版本使用的每线程缓存。

To ensure that TCMalloc can use rseq to enable per-CPU caches, you can disable glibc’s registration of a rseq structure. To disable glibc rseq, set the following environment variable before you start mongod:为了确保TCMalloc可以使用rseq来启用每CPU缓存,您可以禁用glibc对rseq结构的注册。要禁用glibc-rseq,请在启动mongod之前设置以下环境变量:

GLIBC_TUNABLES=glibc.pthread.rseq=0
export GLIBC_TUNABLES

Check Kernel Version检查内核版本

If you disabled glibc rseq and per-CPU caches are still not enabled, ensure that you're using Linux kernel version 4.18 or later. To check your kernel version, run the following command:如果您禁用了glibc-rseq,但每个CPU的缓存仍未启用,请确保您使用的是Linux内核版本4.18或更高版本。要检查内核版本,请运行以下命令:

uname -r

Learn More了解更多