Docs HomeMongoDB Manual

Rolling Index Builds on Sharded Clusters滚动指数建立在分片集群上

Index builds can impact sharded cluster performance. 索引构建可能会影响分片集群的性能。By default, MongoDB 4.4 and later build indexes simultaneously on all data-bearing replica set members. 默认情况下,MongoDB 4.4及更高版本会在所有承载数据的副本集成员上同时构建索引。Index builds on sharded clusters occur only on those shards which contain data for the collection being indexed. 基于分片集群的索引构建仅发生在那些包含要索引的集合的数据的分片上。For workloads which cannot tolerate performance decrease due to index builds, consider using the following procedure to build indexes in a rolling fashion.对于无法容忍由于索引构建而导致性能下降的工作负载,请考虑使用以下过程以滚动方式构建索引。

Rolling index builds take at most one shard replica set member out at a time, starting with the secondary members, and builds the index on that member as a standalone. 滚动索引构建一次最多从辅助成员开始取出一个分片副本集成员,并在该成员上作为独立成员构建索引。Rolling index builds require at least one replica set election per shard.滚动索引构建要求每个分片至少有一个副本集选举。

Considerations注意事项

Unique Indexes唯一索引

To create unique indexes using the following procedure, you must stop all writes to the collection during this procedure.若要使用以下过程创建唯一索引,必须在此过程中停止对集合的所有写入操作。

If you cannot stop all writes to the collection during this procedure, do not use the procedure on this page. 如果在此过程中无法停止对集合的所有写入,请不要使用此页上的过程。Instead, build your unique index on the collection by issuing db.collection.createIndex() on the mongos for a sharded cluster.相反,通过在分片集群的mongos上发出db.collection.createIndex(),在集合上构建唯一索引。

Oplog Size大小

Ensure that your oplog is large enough to permit the indexing or re-indexing operation to complete without falling too far behind to catch up. 确保oplog足够大,可以完成索引或重新索引操作,而不会落后太多。See the oplog sizing documentation for additional information.有关其他信息,请参阅oplog大小调整文档。

Prerequisites先决条件

For building unique indexes用于建立唯一索引
  1. To create unique indexes using the following procedure, you must stop all writes to the collection during the index build. 若要使用以下过程创建唯一索引,必须在索引生成过程中停止对集合的所有写入操作。Otherwise, you may end up with inconsistent data across the replica set members. If you cannot stop all writes to the collection, do not use the following procedure to create unique indexes.否则,复制副本集成员之间的数据可能会不一致。如果无法停止对集合的所有写入,请不要使用以下过程创建唯一索引。

    Warning

    If you cannot stop all writes to the collection, do not use the following procedure to create unique indexes.如果无法停止对集合的所有写入,请不要使用以下过程创建唯一索引。

  2. Before creating the index, validate that no documents in the collection violate the index constraints. 在创建索引之前,请验证集合中是否没有文档违反索引约束。If a collection is distributed across shards and a shard contains a chunk with duplicate documents, the create index operation may succeed on the shards without duplicates but not on the shard with duplicates. 如果一个集合分布在多个分片之间,并且一个分片包含一个具有重复文档的chunk,那么创建索引操作可能会在没有重复文档的分片上成功,但在有重复文档的片段上不会成功。To avoid leaving inconsistent indexes across shards, you can issue the db.collection.dropIndex() from a mongos to drop the index from the collection.为了避免在分片之间留下不一致的索引,您可以从mongos发出db.collection.dropIndex()来从集合中删除索引。

Procedure过程

Important

The following procedure to build indexes in a rolling fashion applies to sharded clusters deployments, and not replica set deployments. For the procedure for replica sets, see Rolling Index Builds on Replica Sets instead.以下以滚动方式构建索引的过程适用于分片集群部署,而不适用于副本集部署。有关复制副本集的过程,请参阅复制副本集上的滚动索引生成

A. Stop the Balancer停止平衡器

Connect mongosh to a mongos instance in the sharded cluster, and run sh.stopBalancer() to disable the balancer: mongosh连接到分片集群中的mongos实例,并运行sh.stopBalancer()禁用平衡器:[1]

sh.stopBalancer()
Note

If a migration is in progress, the system will complete the in-progress migration before stopping the balancer.如果正在进行迁移,系统将在停止平衡器之前完成正在进行的迁移。

To verify that the balancer is disabled, run sh.getBalancerState(), which returns false if the balancer is disabled:要验证平衡器是否已禁用,请运行sh.getBalancerState(),如果平衡器已禁用,则返回false

sh.getBalancerState()
[1] Starting in MongoDB 6.1, automatic chunk splitting is not performed. This is because of balancing policy improvements. 从MongoDB 6.1开始,不执行自动区块分割。这是因为平衡政策的改进。Auto-splitting commands still exist, but do not perform an operation. For details, see Balancing Policy Changes.自动拆分命令仍然存在,但不执行操作。有关详细信息,请参阅平衡策略更改In MongoDB versions earlier than 6.1, sh.stopBalancer() also disables auto-splitting for the sharded cluster.在6.1之前的MongoDB版本中,sh.stopBalancer()还禁用了分片集群的自动拆分。

B. Determine the Distribution of the Collection确定集合的分布

From mongosh connected to the mongos, refresh the cached routing table for that mongos to avoid returning stale distribution information for the collection. 从连接到mongosmongosh,刷新该mongos缓存的路由表,以避免返回集合的过时分发信息。Once refreshed, run db.collection.getShardDistribution() for the collection you wish to build the index.刷新后,对要构建索引的集合运行db.collection.getShardDistribution()

For example, if you want to create an ascending index on the records collection in the test database:例如,如果要在test数据库中的records集合上创建升序索引:

db.adminCommand( { flushRouterConfig: "test.records" } );
db.records.getShardDistribution();

The method outputs the shard distribution. For example, consider a sharded cluster with 3 shards shardA, shardB, and shardC and the db.collection.getShardDistribution() returns the following:该方法输出分片分布。例如,考虑一个具有3个分片shardAshardBshardC的分片集群,db.collection.getShardDistribution()返回以下内容:

Shard shardA at shardA/s1-mongo1.example.net:27018,s1-mongo2.example.net:27018,s1-mongo3.example.net:27018
data : 1KiB docs : 50 chunks : 1
estimated data per chunk : 1KiB
estimated docs per chunk : 50

Shard shardC at shardC/s3-mongo1.example.net:27018,s3-mongo2.example.net:27018,s3-mongo3.example.net:27018
data : 1KiB docs : 50 chunks : 1
estimated data per chunk : 1KiB
estimated docs per chunk : 50

Totals
data : 3KiB docs : 100 chunks : 2
Shard shardA contains 50% data, 50% docs in cluster, avg obj size on shard : 40B
Shard shardC contains 50% data, 50% docs in cluster, avg obj size on shard : 40B

From the output, you only build the indexes for test.records on shardA and shardC.从输出中,您只为shardAshardC上的test.records构建索引。

C. Build Indexes on the Shards That Contain Collection Chunks在包含集合区块的分片上建立索引

For each shard that contains chunks for the collection, follow the procedure to build the index on the shard.对于包含集合块的每个分片,请按照以下过程在该分片上构建索引。

C1. Stop One Secondary and Restart as a Standalone停止一个辅助设备并作为独立设备重新启动

For an affected shard, stop the mongod process associated with one of its secondary. Restart after making the following configuration updates:对于受影响的分片,停止与其一个次要分片关联的mongod进程。进行以下配置更新后重新启动:

If you are using a configuration file, make the following configuration updates:如果您正在使用配置文件,请进行以下配置更新:

For example, for a shard replica set member, the updated configuration file will include content like the following example:例如,对于分片副本集成员,更新后的配置文件将包括如下内容:

net:
bindIp: localhost,<hostname(s)|ip address(es)>
port: 27218
# port: 27018
#replication:
# replSetName: shardA
#sharding:
# clusterRole: shardsvr
setParameter:
skipShardingConfigurationChecks: true
disableLogicalSessionCacheRefresh: true

And restart:然后重新启动:

mongod --config <path/To/ConfigFile>

Other settings (e.g. storage.dbPath, etc.) remain the same.其他设置(例如storage.dbPath等)保持不变。

If using command-line options, make the following configuration updates:如果使用命令行选项,请进行以下配置更新:

For example, restart your shard replica set member without the --replSet and --shardsvr options. 例如,在不使用--replSet--shardsvr选项的情况下重新启动您的分片复制集成员。Specify a new port number and set both the skipShardingConfigurationChecks and disableLogicalSessionCacheRefresh parameters to true:指定一个新的端口号,并将skipShardingConfigurationChecksdisableLogicalSessionCacheRefresh参数都设置为true

mongod --port 27218 --setParameter skipShardingConfigurationChecks=true --setParameter disableLogicalSessionCacheRefresh=true

Other settings (e.g. --dbpath, etc.) remain the same.其他设置(例如--dbpath等)保持不变。

[2](1, 2) By running the mongod on a different port, you ensure that the other members of the replica set and all clients will not contact the member while you are building the index.通过在不同的端口上运行mongod,可以确保副本集的其他成员和所有客户端在构建索引时不会与该成员联系。

C2. Build the Index建立索引

Connect directly to the mongod instance running as a standalone on the new port and create the new index for this instance.直接连接到新端口上作为独立运行的mongod实例,并为此实例创建新索引。

For example, connect mongosh to the instance, and use the db.collection.createIndex() method to create an ascending index on the username field of the records collection:例如,将mongosh连接到实例,并使用db.collection.createIndex()方法在records集合的username字段上创建升序索引:

db.records.createIndex( { username: 1 } )

C3. Restart the Program mongod as a Replica Set Member以副本集成员身份重新启动程序mongod

When the index build completes, shutdown the mongod instance. 当索引构建完成时,关闭mongod实例。Undo the configuration changes made when starting as a standalone to return to its original configuration and restart.撤消以独立方式启动时所做的配置更改,以返回到其原始配置并重新启动。

Important

Be sure to remove the skipShardingConfigurationChecks parameter and disableLogicalSessionCacheRefresh parameter.请确保删除skipShardingConfigurationChecks参数和disableLogicalSessionCacheRefresh参数。

For example, to restart your replica set shard member:例如,要重新启动复制集分片成员:

If you are using a configuration file:如果您正在使用配置文件:

net:
bindIp: localhost,<hostname(s)|ip address(es)>
port: 27018
replication:
replSetName: shardA
sharding:
clusterRole: shardsvr

Other settings (e.g. storage.dbPath, etc.) remain the same.其他设置(例如storage.dbPath等)保持不变。

And restart:然后重新启动:

mongod --config <path/To/ConfigFile>

If you are using command-line options:如果使用命令行选项:

For example:例如:

mongod --port 27018 --replSet shardA --shardsvr

Other settings (e.g. --dbpath, etc.) remain the same.其他设置(例如--dbpath等)保持不变。

Allow replication to catch up on this member.允许复制赶上此成员。

C4. Repeat the Procedure for the Remaining Secondaries for the Shard对分片的剩余次要任务重复此过程

Once the member catches up with the other members of the set, repeat the procedure one member at a time for the remaining secondary members for the shard:一旦成员赶上了集合中的其他成员,就对分片的其余次要成员重复该过程,每次一个成员:

  1. C1. Stop One Secondary and Restart as a Standalone停止一个辅助设备并作为独立设备重新启动
  2. C2. Build the Index建立索引
  3. C3. Restart the Program mongod as a Replica Set Member以副本集成员身份重新启动程序mongod

C5. Build the Index on the Primary在Primary上构建索引

When all the secondaries for the shard have the new index, step down the primary for the shard, restart it as a standalone using the procedure described above, and build the index on the former primary:当分片的所有辅助文件都有新索引时,请关闭分片的主文件,使用上述过程将其作为独立文件重新启动,并在前一个主文件上构建索引:

  1. Use the rs.stepDown() method in mongosh to step down the primary. Upon successful stepdown, the current primary becomes a secondary and the replica set members elect a new primary.使用mongosh中的rs.stepDown()方法来逐步减少初级。成功降级后,当前主副本将变为辅助副本,副本集成员将选择一个新的主副本。
  2. C1. Stop One Secondary and Restart as a Standalone停止一个辅助设备并作为独立设备重新启动
  3. C2. Build the Index建立索引
  4. C3. Restart the Program mongod as a Replica Set Member以副本集成员身份重新启动程序mongod

D. Repeat for the Other Affected Shards对其他受影响的分片重复此步骤

Once you finish building the index for a shard, repeat C. Build Indexes on the Shards That Contain Collection Chunks for the other affected shards.为一个分片构建完索引后,针对其他受影响的分片重复C.在包含集合块的分片上构建索引

E. Restart the Balancer重新启动平衡器

Once you finish the rolling index build for the affected shards, restart the balancer.完成受影响分片的滚动索引构建后,重新启动平衡器。

Connect mongosh to a mongos instance in the sharded cluster, and run sh.startBalancer(): mongosh连接到分片集群中的mongos实例,然后运行sh.startBalancer()[3]

sh.startBalancer()
[3] Starting in MongoDB 6.1, automatic chunk splitting is not performed. 从MongoDB 6.1开始,不执行自动区块分割。This is because of balancing policy improvements. Auto-splitting commands still exist, but do not perform an operation. 这是因为平衡政策的改进。自动拆分命令仍然存在,但不执行操作。For details, see Balancing Policy Changes.有关详细信息,请参阅平衡策略更改In MongoDB versions earlier than 6.1, sh.startBalancer() also enables auto-splitting for the sharded cluster.在6.1之前的MongoDB版本中,sh.startBalancer()还支持对分片集群进行自动拆分。

Additional Information附加信息

A sharded collection has an inconsistent index if the collection does not have the exact same indexes (including the index options) on each shard that contains chunks for the collection. 如果一个分片集合在包含该集合块的每个分片上没有完全相同的索引(包括索引选项),则该集合具有不一致的索引。Although inconsistent indexes should not occur during normal operations, inconsistent indexes can occur, such as:尽管在正常操作期间不应出现不一致的索引,但也可能出现不一致索引,例如:

  • When a user is creating an index with a unique key constraint and one shard contains a chunk with duplicate documents. 当用户创建一个具有unique键约束的索引,并且一个分片包含一个具有重复文档的块时。In such cases, the create index operation may succeed on the shards without duplicates but not on the shard with duplicates.在这种情况下,创建索引操作可能会在没有重复的分片上成功,但在有重复的分片中不会成功。
  • When a user is creating an index across the shards in a rolling manner but either fails to build the index for an associated shard or incorrectly builds an index with different specification.当用户以滚动方式在分片之间创建索引,但未能为关联的分片构建索引或错误地构建了不同规范的索引时。

Starting in MongoDB 4.4 (and 4.2.6), the config server primary periodically checks for index inconsistencies across the shards for sharded collections. 从MongoDB 4.4(和4.2.6)开始,配置服务器主服务器定期检查分片集合的分片之间的索引不一致。To configure these periodic checks, see enableShardedIndexConsistencyCheck and shardedIndexConsistencyCheckIntervalMS.要配置这些定期检查,请参阅enableShardedIndexConsistencyCheckshardedIndexConsistencyCheckIntervalMS

The command serverStatus returns the field shardedIndexConsistency to report on index inconsistencies when run on the config server primary.当在配置服务器主服务器上运行时,命令serverStatus返回字段shardedIndexConsistency以报告索引不一致。

To check if a sharded collection has inconsistent indexes, see Find Inconsistent Indexes Across Shards.若要检查分片集合是否具有不一致的索引,请参阅查找跨分片的不一致索引