Database Manual / Self-Managed Deployments / Administration / Performance

UNIX ulimit Settings for Self-Managed Deployments设置用于自主管理的部署

Most UNIX-like operating systems, including Linux and macOS, provide ways to limit and control the usage of system resources such as threads, files, and network connections on a per-process and per-user basis. 大多数类UNIX操作系统,包括Linux和macOS,都提供了在每个进程和每个用户的基础上限制和控制系统资源(如线程、文件和网络连接)使用的方法。These "ulimits" prevent single users from using too many system resources. Sometimes, these limits have low default values that can cause a number of issues in the course of normal MongoDB operation.这些“ulimits”可以防止单个用户使用太多的系统资源。有时,这些限制的默认值较低,在正常的MongoDB操作过程中可能会导致许多问题。

Resource Utilization资源利用

mongod and mongos each use threads and file descriptors to track connections and manage internal operations. mongodmongos都使用线程和文件描述符来跟踪连接和管理内部操作。This section outlines the general resource utilization patterns for MongoDB. Use these figures in combination with the actual information about your deployment and its use to determine ideal ulimit settings.本节概述了MongoDB的一般资源利用模式。将这些数字与有关部署及其使用的实际信息结合使用,以确定理想的ulimit设置。

Generally, all mongod and mongos instances:一般来说,所有mongodmongos实例:

  • track each incoming connection with two file descriptors and a thread.使用两个文件描述符和一个线程跟踪每个传入连接。
  • track each internal thread or pthread as a system process.将每个内部线程或pthread作为系统进程进行跟踪。

mongod

  • 1 file descriptor for each data file in use by the mongod instance.
  • 1 file descriptor for each journal file used by the mongod instance.
  • In replica sets, each mongod maintains a connection to all other members of the set.

mongod uses background threads for a number of internal processes, including TTL collections, replication, and replica set health checks, which may require a small number of additional resources.

mongos

In addition to the threads and file descriptors for client connections, mongos must maintain connections to all config servers and all shards, which includes all members of all replica sets.

For mongos, consider the following behaviors:

  • mongos instances maintain a connection pool to each shard so that the mongos can reuse connections and quickly fulfill requests without needing to create new connections.
  • You can limit the number of incoming connections using the net.maxIncomingConnections run-time option. By restricting the number of incoming connections you can prevent a cascade effect where the mongos creates too many connections on the mongod instances.

Review and Set Resource Limits审查并设置资源限制

ulimit

You can use the ulimit command at the system prompt to check system limits, as in the following example:您可以在系统提示下使用ulimit命令检查系统限制,如下例所示:

$ ulimit -a
-t: cpu time (seconds) unlimited
-f: file size (blocks) unlimited
-d: data seg size (kbytes) unlimited
-s: stack size (kbytes) 8192
-c: core file size (blocks) 0
-m: resident set size (kbytes) unlimited
-u: processes 64000
-n: file descriptors 64000
-l: locked-in-memory size (kb) unlimited
-v: address space (kb) unlimited
-x: file locks unlimited
-i: pending signals 192276
-q: bytes in POSIX msg queues 819200
-e: max nice 30
-r: max rt priority 65
-N 15: unlimited

ulimit refers to the per-user limitations for various resources. 是指各种资源的每个用户的限制。Therefore, if your mongod instance executes as a user that is also running multiple processes, or multiple mongod processes, you might see contention for these resources. Also, be aware that the processes value (i.e. -u) refers to the combined number of distinct processes and sub-process threads.

On Linux, you can change ulimit settings by issuing a command in the following form:在Linux上,您可以通过发出以下形式的命令来更改ulimit设置:

ulimit -n <value>

There are both "hard" and the "soft" ulimit s that affect MongoDB's performance. The "hard" ulimit refers to the maximum number of processes that a user can have active at any time. This is the ceiling: no non-root process can increase the "hard" ulimit. In contrast, the "soft" ulimit is the limit that is actually enforced for a session or process, but any process can increase it up to "hard" ulimit maximum.

A low "soft" ulimit can cause can't create new thread, closing connection errors if the number of connections grows too high. For this reason, it is extremely important to set both ulimit values to the recommended values.

ulimit will modify both "hard" and "soft" values unless the -H or -S modifiers are specified when modifying limit values.

For many distributions of Linux you can change values by substituting the -n option for any possible value in the output of ulimit -a.

After changing the ulimit settings, you must restart the process to take advantage of the modified settings. On Linux, you can use the /proc file system to see the current limitations on a running process.

Depending on your system's configuration, and default settings, any change to system limits made using ulimit may revert following a system restart. Check your distribution and operating system documentation for more information.根据系统的配置和默认设置,使用ulimit对系统限制所做的任何更改都可能在系统重新启动后恢复。有关更多信息,请查看发行版和操作系统文档。

You should typically start mongod using systemctl, which uses the ulimit settings:您通常应该使用systemctl启动mongod,它使用ulimit设置:

systemctl start mongod.service

If you do not start mongod using systemctl, systemd overrides some of the ulimit settings. For example, if you start mongod as shown in the following command, then a user slice (such as user-1000.slice) systemd setting is used:

mongod --config ~/mongod.conf

Note

A systemd user slice limits the resources for the user's processes.systemd用户分片限制了用户进程的资源。

macOS

For macOS systems that have installed MongoDB Community using the brew installation method, the recommended open files value is automatically set when you start MongoDB through brew services.对于使用brew安装方法安装了MongoDB Community的macOS系统,当您通过brew服务启动MongoDB时,会自动设置推荐的open files值。

For macOS systems running MongoDB Enterprise or using the TGZ installation method, use the ulimit command to set the recommended values. See your operating system documentation for the precise procedure for changing system limits on running systems.

Red Hat Linux Enterprise Server and CentOS红帽Linux企业服务器和CentOS

Red Hat Enterprise Linux and CentOS 6 and 7 enforce a separate max process limitation, nproc, which overrides ulimit settings. This value is defined in the following configuration file, depending on version:

VersionValueFile
RHEL / CentOS 74096/etc/security/limits.d/20-nproc.conf
RHEL / CentOS 61024/etc/security/limits.d/90-nproc.conf

To configure an nproc value for these versions, create a file named /etc/security/limits.d/99-mongodb-nproc.conf with new soft nproc and hard nproc values to increase the process limit. For recommended values, see Recommended ulimit Settings.

With RHEL / CentOS 8, separate nproc values are no longer necessary. The ulimit command is sufficient to configure the required max process values on RHEL / CentOS 8.

Recommended ulimit Settings

Every deployment may have unique requirements and settings; however, the following thresholds and settings are particularly important for mongod and mongos deployments:

  • -f (file size): unlimited
  • -t (cpu time): unlimited
  • -v (virtual memory): unlimited [1]
  • -l (locked-in-memory size): unlimited
  • -n (open files): 64000
  • -m (memory size): unlimited [1] [2]
  • -u (processes/threads): 64000

Always remember to restart your mongod and mongos instances after changing the ulimit settings to ensure that the changes take effect.

Considerations注意事项

  • Incoming connections to a mongod or mongos instance require two file descriptors.mongodmongos实例的传入连接需要两个文件描述符。
  • For the macOS platform, the recommended process limit is 2500, which is the maximum configurable value for this platform.对于macOS平台,建议的进程限制为2500,这是此平台的最大可配置值。

Linux distributions using Upstart使用Upstart的Linux发行版

For Linux distributions that use Upstart, you can specify limits within service scripts if you start mongod and/or mongos instances as Upstart services. You can do this by using limit stanzas.

Specify the Recommended ulimit Settings, as in the following example:

limit fsize unlimited unlimited    # (file size)
limit cpu unlimited unlimited # (cpu time)
limit as unlimited unlimited # (virtual memory size)
limit memlock unlimited unlimited # (locked-in-memory size)
limit nofile 64000 64000 # (open files)
limit nproc 64000 64000 # (processes/threads)

Each limit stanza sets the "soft" limit to the first value specified and the "hard" limit to the second.

After changing limit stanzas, ensure that the changes take effect by restarting the application services, using the following form:更改limit节后,使用以下表单重新启动应用程序服务以确保更改生效:

restart <service name>

Linux distributions using systemd使用systemd的Linux发行版

If you start a mongod and/or mongos instance as a systemd service, you can specify limits within the [Service] section of its service file. The service file has a location like /etc/systemd/system/<process-name>.service.

You can set limits by using resource limit directives.

Specify the Recommended ulimit Settings, as in the following example:

[Service]
# Other directives omitted
# (file size)
LimitFSIZE=infinity
# (cpu time)
LimitCPU=infinity
# (virtual memory size)
LimitAS=infinity
# (locked-in-memory size)
LimitMEMLOCK=infinity
# (open files)
LimitNOFILE=64000
# (processes/threads)
LimitNPROC=64000

Each systemd limit directive sets both the "hard" and "soft" limits to the value specified.

After changing limit stanzas, ensure that the changes take effect by restarting the application services, using the following form:更改limit节后,使用以下表单重新启动应用程序服务以确保更改生效:

systemctl restart <service name>

Note

If you installed MongoDB via a package manager such as yum or apt, the service file installed as part of your installation already contains these ulimit values.如果你通过yumapt等包管理器安装了MongoDB,那么作为安装的一部分安装的服务文件已经包含了这些ulimit值。

/proc File System

Note

This section applies only to Linux operating systems.本节仅适用于Linux操作系统。

The /proc file-system stores the per-process limits in the file system object located at /proc/<pid>/limits, where <pid> is the process's PID or process identifier. You can use the following bash function to return the content of the limits object for a process or processes with a given name:

return-limits(){

for process in $@; do
process_pids=`ps -C $process -o pid --no-headers | cut -d " " -f 2`

if [ -z $@ ]; then
echo "[no $process running]"
else
for pid in $process_pids; do
echo "[$process #$pid -- limits]"
cat /proc/$pid/limits
done
fi

done

}

You can copy and paste this function into a current shell session or load it as part of a script. Call the function with one the following invocations:您可以将此函数复制并粘贴到当前的shell会话中,或将其作为脚本的一部分加载。使用以下调用之一调用函数:

return-limits mongod
return-limits mongos
return-limits mongod mongos
[1](1, 2) If you limit virtual or resident memory size on a system running MongoDB the operating system will refuse to honor additional allocation requests.如果你限制运行MongoDB的系统上的虚拟或驻留内存大小,操作系统将拒绝接受额外的分配请求。
[2] The -m parameter to ulimit has no effect on Linux systems with kernel versions more recent than 2.4.30. You may omit -m if you wish.