Database Manual / Indexes

Index Builds on Populated Collections索引建立在受欢迎的集合之上

Index builds use an optimized build process that holds an exclusive lock on the collection at the beginning and end of the index build. The rest of the build process yields to interleaving read and write operations. For a detailed description of index build process and locking behavior, see Index Build Process.索引构建使用优化的构建过程,该过程在索引构建的开始和结束时对集合进行独占锁定。构建过程的其余部分让位于交替的读写操作。有关索引构建过程和锁定行为的详细说明,请参阅索引构建过程

Index builds on a replica set or sharded cluster build simultaneously across all data-bearing replica set members. The primary requires a minimum number of data-bearing voting members (i.e. commit quorum), including itself, that must complete the build before marking the index as ready for use. 索引基于副本集构建,或在所有承载数据的副本集成员之间同时构建分片集群。初选需要最少数量的数据承载投票成员(即提交法定人数),包括其本身,这些成员必须在将索引标记为可供使用之前完成构建。A "voting" member is any replica set member where members[n].votes is greater than 0. See Index Builds in Replicated Environments for more information.“投票”成员是指members[n].votes大于0的任何副本集成员。有关更多信息,请参阅复制环境中的索引构建。

Starting in MongoDB 7.1, index builds are improved with faster error reporting and increased failure resilience. You can also set the minimum available disk space required for index builds using the new indexBuildMinAvailableDiskSpaceMB parameter, which stops index builds if disk space is too low.从MongoDB 7.1开始,索引构建得到了改进,错误报告更快,故障恢复能力更强。您还可以使用新的indexBuildMinAvailableDiskSpaceMB参数设置索引构建所需的最小可用磁盘空间,如果磁盘空间太低,该参数将停止索引构建。

The following table compares the index build behavior starting in MongoDB 7.1 with earlier versions.下表将MongoDB 7.1中开始的索引构建行为与早期版本进行了比较。

Behavior Starting in MongoDB 7.1从MongoDB 7.1开始的行为Behavior in Earlier MongoDB Versions早期MongoDB版本中的行为
Index errors found during the collection scan phase, except duplicate key errors, are returned immediately and then the index build stops. Earlier MongoDB versions return errors in the commit phase, which occurs near the end of the index build. MongoDB 7.1 helps you to rapidly diagnose index errors. For example, if an incompatible index value format is found, the error is returned to you immediately.在集合扫描阶段发现的索引错误(重复键错误除外)会立即返回,然后索引构建停止。早期的MongoDB版本在提交阶段返回错误,该阶段发生在索引构建接近结束时。MongoDB 7.1可帮助您快速诊断索引错误。例如,如果发现不兼容的索引值格式,则会立即向您返回错误。Index build errors can take a long time to be returned compared to MongoDB 7.1 because the errors are returned near the end of the index build in the commit phase.与MongoDB 7.1相比,索引构建错误可能需要很长时间才能返回,因为错误是在提交阶段索引构建接近结束时返回的。
Increased resilience for your deployment. If an index build error occurs, a secondary member can request that the primary member stop an index build and the secondary member does not crash. 增强部署的弹性。如果发生索引构建错误,secondary成员可以请求primary成员停止索引构建,并且辅助成员不会崩溃。A request to stop an index build is not always possible: if a member has already voted to commit the index, then the secondary cannot request that the index build stop and the secondary crashes (similar to MongoDB 7.0 and earlier).停止索引构建的请求并不总是可能的:如果一个成员已经投票提交了索引,那么辅助服务器就不能请求停止索引构建,并且辅助服务器崩溃(类似于MongoDB 7.0及更早版本)。An index build error can cause a secondary member to crash.索引构建错误可能会导致辅助成员崩溃。
Improved disk space management for index builds. An index build may be automatically stopped if the available disk space is below the minimum specified in the indexBuildMinAvailableDiskSpaceMB parameter. 改进了索引构建的磁盘空间管理。如果可用磁盘空间低于indexBuildMinAvailableDiskSpaceMB参数中指定的最小值,则索引构建可能会自动停止。If a member has already voted to commit the index, then the index build is not stopped.如果成员已经投票提交索引,则索引构建不会停止。An index build is not stopped if there is insufficient available disk space.如果可用磁盘空间不足,索引构建不会停止。

Note

For information about creating indexes in Atlas, refer to the index management page in the Atlas documentation.有关在Atlas中创建索引的信息,请参阅Atlas文档中的索引管理页面。

Behavior行为

Comparison to Foreground and Background Builds与前景和背景建筑的比较

Previous versions of MongoDB supported building indexes either in the foreground or background. Foreground index builds were fast and produced more efficient index data structures, but required blocking all read-write access to the parent database of the collection being indexed for the duration of the build. Background index builds were slower and had less efficient results, but allowed read-write access to the database and its collections during the build process.MongoDB的早期版本支持在前台或后台构建索引。前台索引构建速度很快,并产生了更高效的索引数据结构,但需要在构建期间阻止对正在索引的集合的父数据库的所有读写访问。后台索引构建速度较慢,结果效率较低,但允许在构建过程中对数据库及其集合进行读写访问。

Index builds now obtain an exclusive lock on only the collection being indexed during the start and end of the build process to protect metadata changes. The rest of the build process uses the yielding behavior of background index builds to maximize read-write access to the collection during the build. Index builds still produce efficient index data structures despite the more permissive locking behavior.索引构建现在仅在构建过程的开始和结束期间对正在索引的集合获得独占锁,以保护元数据更改。构建过程的其余部分使用后台索引构建的屈服行为,以在构建过程中最大限度地提高对集合的读写访问。尽管有更宽松的锁定行为,索引构建仍然会产生高效的索引数据结构。

The optimized index build performance is at least on par with background index builds. For workloads with few or no updates received during the build process, optimized index builds can be as fast as a foreground index build on that same data.优化的索引构建性能至少与后台索引构建相当。对于在构建过程中很少或没有收到更新的工作负载,优化的索引构建可以像在相同数据上构建前台索引一样快。

Use db.currentOp() to monitor the progress of ongoing index builds.使用db.currentOp()监视正在进行的索引构建的进度。

MongoDB ignores the background index build option if specified to createIndexes or its shell helpers createIndex() and createIndexes().如果指定了createIndexes或其shell助手createIndex()createIndexes(),MongoDB将忽略background索引构建选项。

Constraint Violations During Index Build索引构建过程中的约束违规

For indexes that enforce constraints on the collection, such as unique indexes, the mongod checks all pre-existing and concurrently-written documents for violations of those constraints after the index build completes. Documents that violate the index constraints can exist during the index build. 对于对集合实施约束的索引,如唯一索引,mongod会在索引构建完成后检查所有预先存在的和并发编写的文档是否违反了这些约束。在索引构建过程中,可能存在违反索引约束的文档。If any documents violate the index constraints at the end of the build, the mongod terminates the build and throws an error.如果任何文档在构建结束时违反了索引约束,mongod将终止构建并抛出错误。

For example, consider a populated collection inventory. An administrator wants to create a unique index on the product_sku field. If any documents in the collection have duplicate values for product_sku, the index build can still start successfully. If any violations still exist at the end of the build, the mongod terminates the build and throws an error.例如,考虑一个已填充的集合inventory。管理员希望在product_sku字段上创建一个唯一索引。如果集合中的任何文档都有重复的product_sku值,索引构建仍然可以成功启动。如果在构建结束时仍然存在任何违规行为,mongod将终止构建并抛出错误。

Similarly, an application can successfully write documents to the inventory collection with duplicate values of product_sku while the index build is in progress. If any violations still exist at the end of the build, the mongod terminates the build and throws an error.同样,在索引构建过程中,应用程序可以成功地将具有重复值product_sku的文档写入inventory集合。如果在构建结束时仍然存在任何违规行为,mongod将终止构建并抛出错误。

To mitigate the risk of index build failure due to constraint violations:为了降低因违反约束而导致的索引构建失败的风险:

  • Validate that no documents in the collection violate the index constraints.验证集合中没有文档违反索引约束。
  • Stop all writes to the collection from applications that cannot guarantee violation-free write operations.停止无法保证无违规写入操作的应用程序对集合的所有写入。

Sharded Collections分片化集合

For a sharded collection distributed across multiple shards, one or more shards may contain a chunk with duplicate documents. As such, the create index operation may succeed on some of the shards (i.e. the ones without duplicates) but not on others (i.e. the ones with duplicates). 对于分布在多个分片上的分片集合,一个或多个分块可能包含一个具有重复文档的块。因此,创建索引操作可能会在某些分片上成功(即没有重复的分片),但在其他分片上可能不会成功(即有重复的分块)。To avoid leaving inconsistent indexes across shards, you can issue the db.collection.dropIndex() from a mongos to drop the index from the collection.为了避免在分片之间留下不一致的索引,您可以从mongos发出db.collection.dropIndex(),从集合中删除索引。

To mitigate the risk of this occurrence, before creating the index:为了降低这种情况发生的风险,在创建索引之前:

  • Validate that no documents in the collection violate the index constraints.验证集合中没有文档违反索引约束。
  • Stop all writes to the collection from applications that cannot guarantee violation-free write operations.停止无法保证无违规写入操作的应用程序对集合的所有写入。

Maximum Concurrent Index Builds最大并发索引构建数

By default, the server allows up to three concurrent index builds. To change the number of allowed concurrent index builds, modify the maxNumActiveUserIndexBuilds parameter.默认情况下,服务器最多允许三个并发索引构建。要更改允许的并发索引生成次数,请修改maxNumActiveUserIndexBuilds参数。

If the number of concurrent index builds reaches the limit specified by maxNumActiveUserIndexBuilds, the server blocks additional index builds until the number of concurrent index builds drops below the limit.如果并发索引构建的数量达到maxNumActiveUserIndexBuilds指定的限制,服务器将阻止其他索引构建,直到并发索引构建数量降至限制以下。

Index Build Impact on Database Performance索引构建对数据库性能的影响

Index Builds During Write-Heavy Workloads在写入繁重工作负载期间建立索引

Building indexes during time periods where the target collection is under heavy write load can result in reduced write performance and longer index builds.在目标集合处于重写负载的时间段内构建索引可能会导致写性能降低和索引构建时间延长。

Consider designating a maintenance window during which applications stop or reduce write operations against the collection. Start the index build during this maintenance window to mitigate the potential negative impact of the build process.考虑指定一个维护窗口,在此期间应用程序停止或减少对集合的写入操作。在此维护窗口期间启动索引构建,以减轻构建过程的潜在负面影响。

Insufficient Available System Memory (RAM)可用系统内存(RAM)不足

createIndexes supports building one or more indexes on a collection. createIndexes uses a combination of memory and temporary files on disk to build indexes. 支持在集合上构建一个或多个索引。createIndexes使用内存和磁盘上的临时文件的组合来构建索引。The default memory limit is 200 megabytes per createIndexes command, shared equally among all indexes built in that command. 默认内存限制为每个createIndexes命令200 MB,在该命令中内置的所有索引之间平均共享。For example, if you build 10 indexes with one createIndexes command, MongoDB allocates each index 20 megabytes for the index build process when using the default memory limit of 200. 例如,如果使用一个createIndexes命令构建10个索引,则MongoDB在使用默认内存限制200时为每个索引分配20 MB用于索引构建过程。When you reach the memory limit, MongoDB creates temporary files in the _tmp subdirectory within --dbpath to complete the build.当达到内存限制时,MongoDB会在--dbpath_tmp子目录中创建临时文件以完成构建。

You can adjust the memory limit with the maxIndexBuildMemoryUsageMegabytes parameter. 您可以使用maxIndexBuildMemoryUsageMegabytes参数调整内存限制。Setting a higher memory limit may result in faster completion of index builds. However, setting this limit too high relative to the unused RAM on your system can result in memory exhaustion and server shutdown.设置更高的内存限制可能会导致更快地完成索引构建。但是,相对于系统上未使用的RAM,将此限制设置得太高可能会导致内存耗尽和服务器关闭。

Each createIndexes command has a limit of maxIndexBuildMemoryUsageMegabytes. 每个createIndexes命令都有maxIndexBuildMemoryUsageMegabytes的限制。When using the default maxNumActiveUserIndexBuilds of 3, the total memory usage for all concurrent index builds can reach up to 3 times the value of maxIndexBuildMemoryUsageMegabytes.当使用默认的maxNumActiveUserIndexBuilds3时,所有并发索引构建的总内存使用量最多可达maxIndexBuildMemoryUsageMegabytes值的3倍。

If the host machine has limited available free RAM, you may need to schedule a maintenance period to increase the total system RAM before you can modify the mongod RAM usage.如果主机的可用RAM有限,您可能需要安排一个维护期来增加总系统RAM,然后才能修改mongod RAM的使用情况。

Index Builds in Replicated Environments在复制环境中构建索引

Note

Requires 需要featureCompatibilityVersion 4.4+

Each mongod in the replica set or sharded cluster must have featureCompatibilityVersion set to at least 4.4 to start index builds simultaneously across replica set members.副本集或分片集群中的每个mongod都必须将featureCompatibilityVersion设置为至少4.4,才能在副本集成员之间同时启动索引构建。

Index builds on a replica set or sharded cluster build simultaneously across all data-bearing replica set members. For sharded clusters, the index build occurs only on shards containing data for the collection being indexed. 索引基于副本集构建,或在所有承载数据的副本集成员之间同时构建分片集群。对于分片集群,索引构建仅发生在包含被索引集合的数据的分片上。The primary requires a minimum number of data-bearing voting members (i.e commit quorum), including itself, that must complete the build before marking the index as ready for use.初选需要最少数量的数据承载voting成员(即提交法定人数),包括其本身,在将索引标记为可供使用之前,必须完成构建。

Important

If a data-bearing voting node becomes unreachable and the commitQuorum is set to the default votingMembers, index builds can hang until that node comes back online.如果承载数据的投票节点变得无法访问,并且commitQuorum设置为默认的votingMembers(投票成员),则索引构建可以挂起,直到该节点重新联机。

The build process is summarized as follows:构建过程总结如下:

  1. The primary receives the createIndexes command and immediately creates a "startIndexBuild" oplog entry associated with the index build.主服务器接收createIndexes命令,并立即创建与索引构建关联的“startIndexBuild”oplog条目。
  2. The secondaries start the index build after they replicate the "startIndexBuild" oplog entry.次级服务器在复制“startIndexBuild”oplog条目后开始索引构建。
  3. Each member "votes" to commit the build once it finishes indexing data in the collection.一旦完成对集合中数据的索引,每个成员都会“投票”提交构建。
  4. Secondary members continue to process any new write operations into the index while waiting for the primary to confirm a quorum of votes.次要成员在等待主要成员确认法定投票人数的同时,继续处理对索引的任何新写入操作。
  5. When the primary has a quorum of votes, it checks for any key constraint violations such as duplicate key errors.当初选达到法定票数时,它会检查是否存在任何违反键约束的情况,如重复键错误。

    • If there are no key constraint violations, the primary completes the index build, marks the index as ready for use, and creates an associated "commitIndexBuild" oplog entry.如果没有违反键约束,主服务器将完成索引构建,将索引标记为可供使用,并创建相关的“commitIndexBuild”oplog条目。
    • If there are any key constraint violations, the index build fails. The primary aborts the index build and creates an associated "abortIndexBuild" oplog entry.如果存在任何违反键约束的情况,则索引构建将失败。主进程中止索引构建并创建相关的“abortIndexBuild”oplog条目。
  6. The secondaries replicate the "commitIndexBuild" oplog entry and complete the index build.次级服务器复制“commitIndexBuild”oplog条目并完成索引构建。

    If the secondaries instead replicate an "abortIndexBuild" oplog entry, they abort the index build and discard the build job.如果次级服务器复制“abortIndexBuild”oplog条目,则会中止索引构建并丢弃构建作业。

For sharded clusters, the index build occurs only on shards containing data for the collection being indexed.对于分片集群,索引构建仅发生在包含被索引集合的数据的分片上。

For a more detailed description of the index build process, see Index Build Process.有关索引构建过程的更详细描述,请参阅索引构建过程

By default, index builds use a commit quorum of "votingMembers", or all data-bearing voting members. To start an index build with a non-default commit quorum, specify the commitQuorum parameter to createIndexes or its shell helpers db.collection.createIndex() and db.collection.createIndexes().默认情况下,索引构建使用"votingMembers"的提交法定人数,或所有携带数据的投票成员。要使用非默认提交仲裁启动索引构建,请将commitQuorum参数指定给createIndexes或其shell助手db.collection.createIndex()db.collection.createIndexes()

To modify the commit quorum required for an in-progress simultaneous index build, use the setIndexCommitQuorum command.要修改正在进行的同步索引构建所需的提交仲裁,请使用setIndexCommitQuorum命令。

Warning

Avoid performing rolling index and replicated index build processes concurrently as it might lead to unexpected issues, such as broken builds and crash loops.避免同时执行滚动索引和复制索引构建过程,因为这可能会导致意外问题,如构建中断和崩溃循环。

Note

Rolling index builds take at most one replica set member at a time, starting with the secondary members, and build the index on that member as a standalone. Rolling index builds require at least one replica set election. 滚动索引构建一次最多只能包含一个副本集成员,从次要成员开始,并在该成员上作为独立对象构建索引。滚动索引构建需要至少一个副本集选择。Rolling index builds should only be used if the customers meet the requirements listed on the rolling index pages as the procedure lowers the resiliency of the cluster.只有当客户满足滚动索引页面上列出的要求时,才应使用滚动索引构建,因为该过程会降低集群的弹性。

Commit Quorum Contrasted with Write Concern提交法定人数与写入关注度的对比

There are important differences between commit quorums and write concerns:提交法定人数写入关注之间存在重要区别:

  • Index builds use commit quorums.索引构建使用提交法定人数。
  • Write operations use write concerns.写操作使用写关注。

Each data-bearing node in a cluster is a voting member.集群中的每个数据承载节点都是一个投票成员。

The commit quorum specifies how many data-bearing voting members, or which voting members, including the primary, must be prepared to commit a simultaneous index build before the primary will execute the commit.提交法定人数指定了在主要成员执行提交之前,必须准备提交多少数据承载投票成员,或哪些投票成员(包括主要成员)来提交同时的索引构建

The write concern is the level of acknowledgment that the write has propagated to the specified number of instances.写入关注是指写入已传播到指定数量的实例的确认级别。

Changed in version 8.0.在版本8.0中的更改。 The commit quorum specifies how many nodes must be ready to finish the index build before the primary commits the index build. In contrast, when the primary has committed the index build, the write concern specifies how many nodes must replicate the index build oplog entry before the command returns success.提交仲裁指定了在主提交索引构建之前,必须有多少节点准备好完成索引构建。相比之下,当主节点提交了索引构建时,写关注指定了在命令返回成功之前必须复制索引构建oplog条目的节点数量。

In previous releases, when the primary committed the index build, the write concern specified how many nodes must finish the index build before the command returned success.在以前的版本中,当主提交索引构建时,写关注指定了在命令返回成功之前必须完成多少节点的索引构建。

Build Failure and Recovery构建失败和恢复

Interrupted Index Builds on a Primary mongod中断索引建立在一个Primarymongod

Starting in MongoDB 5.0, if the primary mongod performs a clean shutdown with "force" : true or receives a SIGTERM signal during an index build and the commitQuorum is set to the default votingMembers, then the index build progress is saved to disk. 从MongoDB 5.0开始,如果主mongod使用"force" : true执行干净shutdown,或者在索引构建过程中收到SIGTERM信号,并且commitQuorum设置为默认votingMembers,则索引构建进度将保存到磁盘。The mongod automatically recovers the index build when it is restarted and continues from the saved checkpoint. In earlier versions, if the index build is interrupted, it has to be restarted from the beginning.mongod在重新启动时自动恢复索引构建,并从保存的检查点继续。在早期版本中,如果索引构建中断,则必须从头重新启动。

Interrupted Index Builds on a Secondary mongod中断索引建立在二级mongod之上

Starting in MongoDB 5.0, if a secondary mongod performs a clean shutdown with "force" : true or receives a SIGTERM signal during an index build and the commitQuorum is set to the default votingMembers, then the index build progress is saved to disk. 从MongoDB 5.0开始,如果辅助mongod使用"force" : true执行干净shutdown,或者在索引构建过程中收到SIGTERM信号,并且commitQuorum设置为默认votingMembers,则索引构建进度将保存到磁盘。The mongod automatically recovers the index build when it is restarted and continues from the saved checkpoint. In earlier versions, if the index build is interrupted, it has to be restarted from the beginning.mongod在重新启动时自动恢复索引构建,并从保存的检查点继续。在早期版本中,如果索引构建中断,则必须从头重新启动。

The mongod can perform the startup process while the recovering index builds.mongod可以在构建恢复索引的同时执行启动过程。

If you restart the mongod as a standalone (i.e. removing or commenting out replication.replSetName or omitting --replSetName), the mongod cannot restart the index build. 如果您将mongod作为独立的重新启动(即删除或注释掉replication.replSetName或省略--replSetName),mongod将无法重新启动索引构建。The build remains in a paused state until it is manually dropped.生成将保持暂停状态,直到手动dropped

Interrupted Index Builds on Standalone mongod基于独立mongod的中断索引构建

If the mongod shuts down during the index build, the index build job and all progress is lost. 如果mongod在索引构建过程中关闭,则索引构建作业和所有进度都将丢失。Restarting the mongod does not restart the index build. You must re-issue the createIndex() operation to restart the index build.重新启动mongod不会重新启动索引构建。您必须重新发出createIndex()操作才能重新启动索引构建。

Rollbacks during Build Process构建过程中的回滚

Starting in MongoDB 5.0, if a node is rolled back to a prior state during the index build, the index build progress is saved to disk. If there is still work to be done when the rollback concludes, the mongod automatically recovers the index build and continues from the saved checkpoint.从MongoDB 5.0开始,如果在索引构建过程中将节点回滚到之前的状态,则索引构建进度将保存到磁盘。如果回滚结束时仍有工作要做,mongod会自动恢复索引构建,并从保存的检查点继续。

MongoDB can pause an in-progress index build to perform a rollback.MongoDB可以暂停正在进行的索引构建以执行回滚

  • If the rollback does not revert the index build, MongoDB restarts the index build after completing the rollback.如果回滚没有还原索引构建,MongoDB将在完成回滚后重新启动索引构建。
  • If the rollback reverts the index build, you must re-create the index or indexes after the rollback completes.如果回滚还原了索引构建,则必须在回滚完成后重新创建一个或多个索引。

Index Consistency Checks for Sharded Collections分片集合的索引一致性检查

A sharded collection has an inconsistent index if the collection does not have the exact same indexes (including the index options) on each shard that contains chunks for the collection. Although inconsistent indexes should not occur during normal operations, inconsistent indexes can occur, such as:如果一个分片集合在包含该集合块的每个分片上没有完全相同的索引(包括索引选项),则该分片集合的索引不一致。虽然在正常操作过程中不应出现不一致的索引,但可能会出现不一致索引,例如:

  • When a user is creating an index with a unique key constraint and one shard contains a chunk with duplicate documents. In such cases, the create index operation may succeed on the shards without duplicates but not on the shard with duplicates.当用户创建具有unique键约束的索引时,一个分片包含一个包含重复文档的块。在这种情况下,创建索引操作可能会在没有重复的分片上成功,但在有重复的分片上则不会成功。
  • When a user is creating an index across the shards in a rolling manner (i.e. manually building the index one by one across the shards) but either fails to build the index for an associated shard or incorrectly builds an index with different specification.当用户以滚动方式(即在分片上逐一手动构建索引)在分片之间创建索引时,但要么无法为相关分片构建索引,要么错误地构建了具有不同规范的索引。

The config server primary periodically checks for index inconsistencies across the shards for sharded collections. 配置服务器主服务器定期检查分片集合的分片索引是否不一致。To configure these periodic checks, see enableShardedIndexConsistencyCheck and shardedIndexConsistencyCheckIntervalMS.要配置这些定期检查,请参阅启用enableShardedIndexConsistencyCheckshardedIndexConsistencyCheckIntervalMS

The command serverStatus returns the field shardedIndexConsistency to report on index inconsistencies when run on the config server primary.命令serverStatus返回字段shardedIndexConsistency,以在配置服务器主服务器上运行时报告索引不一致。

To check if a sharded collection has inconsistent indexes, see Find Inconsistent Indexes Across Shards.要检查分片集合是否具有不一致的索引,请参阅跨分片查找不一致索引

Monitor In Progress Index Builds监控进行中索引构建

To see the status of an index build operation, you can use the db.currentOp() method in mongosh. To filter the current operations for index creation operations, see Active Indexing Operations for an example.要查看索引构建操作的状态,可以使用mongosh中的db.currentOp()方法。要筛选索引创建操作的当前操作,请参阅活动索引操作以获取示例。

The msg field includes a percentage-complete measurement of the current stage in the index build process.msg字段包括索引构建过程中当前阶段的完成百分比度量。

Observe Stopped and Resumed Index Builds in the Logs观察日志中已停止和已恢复的索引构建

While an index is being built, progress is written to the MongoDB log. If an index build is stopped and resumed there will be log messages with fields like these:在构建索引的同时,进度会写入MongoDB日志。如果停止并恢复索引构建,将出现包含以下字段的日志消息:

 "msg":"Index build: wrote resumable state to disk",

"msg":"Found index from unfinished build",

Terminate In Progress Index Builds终止进行中的索引构建

Use the dropIndexes command or its shell helpers dropIndex() or dropIndexes() to terminate an in-progress index build. 使用dropIndexes命令或其shell助手dropIndex()dropIndexes()终止正在进行的索引构建。See Stop In-Progress Index Builds for more information.有关更多信息,请参阅正在构建的停止索引

Do not use killOp to terminate an in-progress index builds in replica sets or sharded clusters.不要使用killOp终止副本集或分片集群中正在进行的索引构建。

Index Build Process索引构建过程

The following table describes each stage of the index build process:下表描述了索引构建过程的每个阶段:

Stage阶段Description描述
LockThe mongod obtains an exclusive X lock on the the collection being indexed. This blocks all read and write operations on the collection, including the application of any replicated write operations or metadata commands that target the collection. mongod在被索引的集合上获得一个独占的X锁。这会阻止对集合的所有读写操作,包括应用任何针对集合的复制写操作或元数据命令。The mongod does not yield this lock.猫鼬不会交出这把锁。
Initialization初始化

The mongod creates three data structures at this initial state:mongod在初始状态下创建了三个数据结构:

  • The initial index metadata entry.初始索引元数据条目。
  • A temporary table ("side writes table") that stores keys generated from writes to the collection being indexed during the build process.一个临时表(“侧写表”),用于存储在构建过程中对被索引的集合进行写入时生成的键。
  • A temporary table ("constraint violation table") for all documents that may cause a key generation error. Key generation errors occur when a document has invalid keys for the indexed fields. 所有可能导致键生成错误的文档的临时表(“约束冲突表”)。当文档的索引字段的键无效时,会发生键生成错误。For example, a document with duplicate field values when building a unique index or malformed GeoJSON objects when building a 2dsphere index.例如,在构建唯一索引时具有重复字段值的文档,或在构建2dsphere索引时具有格式错误的GeoJSON对象
LockThe mongod downgrades the exclusive X collection lock to an intent exclusive IX lock. The mongod periodically yields this lock to interleaving read and write operations.mongod将独占X集合锁降级为意图独占IX锁。mongod会定期向交错的读写操作提供此锁。
Scan Collection扫描集合

For each document in the collection, the mongod generates a key for that document and dumps the key into an external sorter.对于集合中的每个文档,mongod都会为该文档生成一个键,并将键转储到外部排序器中。

If the mongod encounters a key generation error while generating a key during the collection scan, it stores that key in the constraint violation table for later processing.如果mongod在集合扫描期间生成键时遇到键生成错误,它会将该键存储在约束冲突表中以供以后处理。

If the mongod encounters any other error while generating a key, the build fails with an error.如果mongod在生成键时遇到任何其他错误,则构建将失败并出现错误。

Once the mongod completes the collection scan, it dumps the sorted keys into the index.一旦mongod完成集合扫描,它就会将排序后的键转储到索引中。

Process Side Writes Table进程侧写入表

The mongod drains the side write table using first-in-first-out priority.mongod使用先进先出的优先级清空侧写表。

If the mongod encounters a key generation error while processing a key in the side write table, it stores that key in the constraint violation table for later processing.如果mongod在处理侧写表中的键时遇到键生成错误,它会将该键存储在约束冲突表中以供以后处理。

If the mongod encounters any other error while processing a key, the build fails with an error.如果mongod在处理键时遇到任何其他错误,则构建将失败并出现错误。

For each document written to the collection during the build process, the mongod generates a key for that document and stores it in the side write table for later processing. The mongod uses a snapshot system to set a limit to the number of keys to process.对于在构建过程中写入集合的每个文档,mongod都会为该文档生成一个键,并将其存储在侧写表中以供以后处理。mongod使用快照系统来设置要处理的键数量的限制。

Vote and Wait for Commit Quorum投票并等待法定人数

A mongod that is not part of a replica set skips this stage.不属于复制品集的猫鼬会跳过此阶段。

The mongod submits a "vote" to the primary to commit the index. Specifically, it writes the "vote" to an internal replicated collection on the primary.mongod向primary提交“投票”以提交索引。具体来说,它将“投票”写入primary上的内部复制集合。

If the mongod is the primary, it waits until it has a commit quorum of votes (all voting data-bearing members by default) before continuing the index build process.如果mongodprimary,它会等待达到提交法定人数的投票(默认情况下是所有投票数据承载的成员),然后再继续索引构建过程。

If the mongod is a secondary, it waits until it replicates either a "commitIndexBuild" or "abortIndexBuild" oplog entry:如果mongodsecondary,它会等待直到复制“commitIndexBuild”或“abortIndexBuild”oplog条目:

  • If the mongod replicates a "commitIndexBuild" oplog entry, it finishes draining the side writes table and moves to the next stage in the index build process.如果mongod复制了“commitIndexBuild”oplog条目,它将完成侧写表的清空,并进入索引构建过程的下一阶段。
  • If the mongod replicates an "abortIndexBuild" oplog entry, it aborts the index build and discards the build job.如果mongod复制了“abortIndexBuild”oplog条目,它将中止索引构建并丢弃构建作业。

While waiting for commit quorum, the mongod adds any additional keys generated from write operations to the collection being indexed to the side writes table and periodically drains the table.在等待提交仲裁时,mongod会将写入操作生成的任何其他键添加到正在索引到侧写入表的集合中,并定期清空该表。

LockThe mongod upgrades the intent exclusive IX lock on the collection to a shared S lock. This blocks all write operations to the collection, including the application of any replicated write operations or metadata commands that target the collection.mongod将集合上的intent独占IX锁升级为共享S锁。这会阻止对集合的所有写入操作,包括应用任何针对集合的复制写入操作或元数据命令。
Finish Processing Temporary Side Writes Table完成临时侧写表的处理

The mongod continues draining remaining records in the side writes table. The mongod may pause replication during this stage.mongod继续清空侧写表中的剩余记录。在这个阶段,猫鼬可能会暂停复制。

If the mongod encounters a key generation error while processing a key in the side write table, it stores that key in the constraint violation table for later processing.如果mongod在处理侧写表中的键时遇到键生成错误,它会将该键存储在约束冲突表中以供以后处理。

If the mongod encounters any other error while processing a key, the build fails with an error.如果mongod在处理键时遇到任何其他错误,则构建将失败并出现错误。

LockThe mongod upgrades the shared S lock on the collection to an exclusive X lock on the collection. This blocks all read and write operations on the collection, including the application of any replicated write operations or metadata commands that target the collection. The mongod does not yield this lock.mongod将集合上的共享S锁升级为集合上的独占X锁。这会阻止对集合的所有读写操作,包括应用任何针对集合的复制写操作或元数据命令。猫鼬不会交出这把锁。
Drop Side Write Table侧边写表

The mongod applies any remaining operations in the side writes table before dropping it.mongod在删除侧写表之前,会应用侧写表中的任何剩余操作。

If the mongod encounters a key generation error while processing a key in the side write table, it stores that key in the constraint violation table for later processing.如果mongod在处理侧写表中的键时遇到键生成错误,它会将该键存储在约束冲突表中以供以后处理。

If the mongod encounters any other error while processing a key, the build fails with an error.如果mongod在处理键时遇到任何其他错误,则构建将失败并出现错误。

At this point, the index includes all data written to the collection.此时,索引包括写入集合的所有数据。

Process Constraint Violation Table违反流程约束表

If the mongod is the primary, it drains the constraint violation table using first-in-first-out priority.如果mongodprimary,它将使用先进先出的优先级清空约束违规表。

  • If no keys in the constraint violation table produce a key generation error or if the table is empty, the mongod drops the table and creates a "commitIndexBuild" oplog entry. Secondaries can complete the associated index build after replicating the oplog entry.如果约束冲突表中没有键产生键生成错误,或者如果表为空,mongod会删除该表并创建一个“commitIndexBuild”oplog条目。借调人员可以在复制oplog条目后完成相关的索引构建。
  • If any key in the constraint violation table still produces a key generation error, the mongod aborts the build and throws an error. The mongod creates an associated "abortIndexBuild" oplog entry to indicate that secondaries should abort and discard the index build job.如果约束冲突表中的任何键仍然产生键生成错误,mongod将中止构建并抛出错误。mongod创建了一个关联的“abortIndexBuild”oplog条目,以指示次级服务器应中止并丢弃索引构建作业。

If the mongod is a secondary, it drops the constraint violation table. 如果mongodsecondary,它会删除约束冲突表。Since the primary must successfully drain the constraint violation table prior to creating the "commitOplogEntry" oplog entry, the secondary can safely assume that no violations exist.由于主服务器必须在创建“commitOplogEntry”oplog条目之前成功地清空约束冲突表,因此辅助服务器可以安全地假设不存在冲突。

Mark the Index as Ready将索引标记为就绪

The mongod updates the index metadata to mark the index as ready for use.mongod会更新索引元数据,将索引标记为可供使用。

LockThe mongod releases the X lock on the collection.mongod释放集合上的X锁。