About this Task关于此任务
Rolling index builds are an alternative to default index builds.滚动索引构建是默认索引构建的替代方案。
Warning
Avoid performing rolling index and replicated index build processes concurrently as it might lead to unexpected issues, such as broken builds and crash loops.避免同时执行滚动索引和复制索引构建过程,因为这可能会导致意外问题,如构建中断和崩溃循环。
Considerations注意事项
Warning
Make sure you are not performing DDL operations while conducting the rolling index build.确保在进行滚动索引构建时没有执行DDL操作。
Unique Indexes唯一索引
To create unique indexes using the following procedure, you must stop all writes to the collection during this procedure.要使用以下过程创建唯一索引,您必须在此过程中停止对集合的所有写入。
If you cannot stop all writes to the collection during this procedure, do not use the procedure on this page. Instead, build your unique index on the collection by issuing 如果在此过程中无法停止对集合的所有写入,请不要使用此页面上的过程。相反,通过在分片集群的db.collection.createIndex() on the mongos for a sharded cluster.mongos上发出db.collection.createIndex(),在集合上构建唯一索引。
Oplog Size操作日志大小
Ensure that your oplog is large enough to permit the indexing or re-indexing operation to complete without falling too far behind to catch up. See the oplog sizing documentation for additional information.确保oplog足够大,以允许索引或重新索引操作完成,而不会落后太多而无法赶上。有关更多信息,请参阅oplog大小文档。
Rolling index builds lower the resiliency of your cluster and increase build duration.滚动索引构建会降低集群的弹性,并增加构建持续时间。
Before You Begin开始之前
For building unique indexes用于构建独特的索引-
To create unique indexes using the following procedure, you must stop all writes to the collection during the index build.要使用以下过程创建唯一索引,您必须在索引构建期间停止对集合的所有写入。Otherwise, you may end up with inconsistent data across the replica set members. If you cannot stop all writes to the collection, do not use the following procedure to create unique indexes.否则,您可能会在副本集成员之间得到不一致的数据。如果无法停止对集合的所有写入,请不要使用以下过程创建唯一索引。Warning
If you cannot stop all writes to the collection, do not use the following procedure to create unique indexes.如果无法停止对集合的所有写入,请不要使用以下过程创建唯一索引。Before creating the index, validate that no documents in the collection violate the index constraints. If a collection is distributed across shards and a shard contains a chunk with duplicate documents, the create index operation may succeed on the shards without duplicates but not on the shard with duplicates.在创建索引之前,请验证集合中没有文档违反索引约束。如果一个集合分布在多个分片上,并且一个分片包含一个具有重复文档的块,则创建索引操作可能会在没有重复的分片上成功,但在有重复的分片上则不会成功。To avoid leaving inconsistent indexes across shards, you can issue the为了避免在分片之间留下不一致的索引,您可以从db.collection.dropIndex()from amongosto drop the index from the collection.mongos发出db.collection.dropIndex(),从集合中删除索引。
Starting in MongoDB 8.0, you can use the 从MongoDB 8.0开始,您可以使用directShardOperations role to perform maintenance operations that require you to execute commands directly against a shard.directShardOperations角色执行维护操作,这些操作要求您直接对分片执行命令。
Warning
Running commands using the 使用directShardOperations role can cause your cluster to stop working correctly and may cause data corruption. Only use the directShardOperations role for maintenance purposes or under the guidance of MongoDB support. directShardOperations角色运行命令可能会导致集群停止正常工作,并可能导致数据损坏。仅将directShardOperations角色用于维护目的或在MongoDB支持的指导下使用。Once you are done performing maintenance operations, stop using the 完成维护操作后,停止使用directShardOperations role.directShardOperations角色。
Procedure过程
Important
The following procedure to build indexes in a rolling fashion applies to sharded clusters deployments, and not replica set deployments. For the procedure for replica sets, see Create a Rolling Index Build on Replica Sets instead.以下以滚动方式构建索引的过程适用于分片集群部署,而不是副本集部署。有关副本集的过程,请参阅在副本集上创建滚动索引构建。
A. Stop Migrations停止迁移
Connect 将mongosh to a mongos instance in the sharded cluster and disable migrations for the collection where you want to perform the rolling build index:mongosh连接到分片集群中的mongos实例,并禁用要执行滚动构建索引的集合的迁移:
db.adminCommand(
{
setAllowMigrations: "<db>.<collection>",
allowMigrations: false
}
)
The preceding command ensures the correct set of shards is targeted for rolling index builds because no migration for the collection will be allowed to commit.前面的命令确保滚动索引构建的目标是正确的分片集,因为不允许提交集合的迁移。
If the command returns the following error, it means the collection is unsharded. You can safely ignore the error and continue with the next step.如果该命令返回以下错误,则表示该集合未被记录。您可以放心地忽略错误并继续下一步。
MongoServerError[NamespaceNotSharded]: Collection must be sharded so migrations can be blockedB. Determine the Distribution of the Collection确定藏品的分布
To determine which shards must be involved in the rolling index build, run the following aggregation on the collection that you want to build the index on:要确定滚动索引构建中必须涉及哪些分片,请在要构建索引的集合上运行以下聚合:
db.getSiblingDB(<db>).getCollection(<collection>).aggregate([{$collStats:{}},{$group: {_id: "$ns", shard_list: {$addToSet: "$shard"}}}])
For example, if you want to create an index on the 例如,如果要在records collection in the test database:test数据库中的records集合上创建索引:
db.getSiblingDB("test").getCollection("records").aggregate([{$collStats:{}},{$group: {_id: "$ns", shard_list: {$addToSet: "$shard"}}}])
[ { _id: 'test.records', shard_list: [ 'shardA', 'shardC' ] } ]
From the output, you only build the indexes for 从输出中,您只需为test.records on shardA and shardC.shardA和shardC上的test.records构建索引。
C. Build Indexes on the Shards That Contain Collection Chunks在包含集合块的分片上构建索引
For each shard that contains chunks for the collection, follow the procedure to build the index on the shard.对于包含集合块的每个分片,按照以下过程在分片上构建索引。
C1. Stop One Secondary and Restart as a Standalone停止一个Secondary并作为单机程序重新启动
For an affected shard, stop the 对于受影响的分片,停止与其secondary一关联的mongod process associated with one of its secondary. Restart after making the following configuration updates:mongod进程。进行以下配置更新后重新启动:
Configuration File配置文件
If you are using a configuration file, make the following configuration updates:如果您正在使用配置文件,请进行以下配置更新:
Change the将net.portto a different port. [1] Make a note of the original port setting as a comment.net.port更改为其他端口。[1]记下原始端口设置作为注释。Comment out the注释掉replication.replSetNameoption.replication.replSetName选项。Comment out the注释掉sharding.clusterRoleoption.sharding.clusterRole选项。Set parameter在skipShardingConfigurationCheckstotruein thesetParametersection.setParameter部分将参数skipShardingConfigurationChecks设置为true。Set parameter在disableLogicalSessionCacheRefreshtotruein thesetParametersection.setParameter节中将参数disableLogicalSessionCacheRefresh设置为true。
For example, for a shard replica set member, the updated configuration file will include content like the following example:例如,对于分片副本集成员,更新后的配置文件将包括以下示例中的内容:
net:
bindIp: localhost,<hostname(s)|ip address(es)>
port: 27218
# port: 27018
#replication:
# replSetName: shardA
#sharding:
# clusterRole: shardsvr
setParameter:
skipShardingConfigurationChecks: true
disableLogicalSessionCacheRefresh: true
And restart:然后重新启动:
mongod --config <path/To/ConfigFile>
Other settings (e.g. 其他设置(如storage.dbPath, etc.) remain the same.storage.dbPath等)保持不变。
Command-line Options命令行选项
If using command-line options, make the following configuration updates:如果使用命令行选项,请进行以下配置更新:
Modify将--portto a different port.--port修改为其他端口。[1]Remove删除--replSet.--replSet。Remove如果是分片成员,则删除--shardsvrif a shard member and--configsvrif a config server member.--shardsvr;如果是配置服务器成员,则移除--configsvr。Set parameter在skipShardingConfigurationCheckstotruein the--setParameteroption.--setParameter选项中将参数skipShardingConfigurationChecks设置为true。Set parameter在disableLogicalSessionCacheRefreshtotruein the--setParameteroption.--setParameter选项中将参数disableLogicalSessionCacheRefresh设置为true。
For example, restart your shard replica set member without the 例如,在不使用--replSet and --shardsvr options. --replSet和--shardsvr选项的情况下重新启动分片副本集成员。Specify a new port number and set both the 指定一个新的端口号,并将skipShardingConfigurationChecks and disableLogicalSessionCacheRefresh parameters to true:skipShardingConfigurationChecks和disableLogicalSessionCacheRefresh参数都设置为true:
mongod --port 27218 --setParameter skipShardingConfigurationChecks=true --setParameter disableLogicalSessionCacheRefresh=true
Other settings (e.g. 其他设置(例如--dbpath, etc.) remain the same.--dbpath等)保持不变。
| [1] | (1, 2) mongod on a different port, you ensure that the other members of the replica set and all clients will not contact the member while you are building the index.mongod,您可以确保副本集的其他成员和所有客户端在构建索引时不会联系到该成员。 |
C2. Build the Index构建索引
Connect directly to the 直接连接到在新端口上作为独立运行的mongod instance running as a standalone on the new port and create the new index for this instance.mongod实例,并为此实例创建新索引。
For example, connect 例如,将mongosh to the instance, and use the db.collection.createIndex() method to create an ascending index on the username field of the records collection:mongosh连接到实例,并使用db.collection.createIndex()方法在记录集合的用户名字段上创建一个升序索引:
db.records.createIndex( { username: 1 } )C3. Restart the Program mongod as a Replica Set Member以副本集成员身份重新启动程序mongod
mongod as a Replica Set MemberWhen the index build completes, shutdown the 索引构建完成后,关闭mongod instance. Undo the configuration changes made when starting as a standalone to return to its original configuration and restart.mongod实例。撤消作为独立设备启动时所做的配置更改,以返回其原始配置并重新启动。
Important
Be sure to remove the 请确保删除skipShardingConfigurationChecks parameter and disableLogicalSessionCacheRefresh parameter.skipShardingConfigurationChecks参数并禁用LogicalSessionCacheRefresh参数。
For example, to restart your replica set shard member:例如,要重新启动副本集分片成员:
Configuration File配置文件
If you are using a configuration file:如果您使用的是配置文件:
Revert to the original port number.恢复到原始端口号。Uncomment the取消对replication.replSetName.replication.replSetName的注释。Uncomment the取消对sharding.clusterRole.sharding.clusterRole的注释。Remove parameter删除skipShardingConfigurationChecksin thesetParametersection.setParameter部分中的参数skipShardingConfigurationChecks。Remove parameter删除disableLogicalSessionCacheRefreshin thesetParametersection.setParameter节中的disableLogicalSessionCacheRefresh参数。
net:
bindIp: localhost,<hostname(s)|ip address(es)>
port: 27018
replication:
replSetName: shardA
sharding:
clusterRole: shardsvr
Other settings (e.g. 其他设置(如storage.dbPath, etc.) remain the same.storage.dbPath等)保持不变。
And restart:然后重新启动:
mongod --config <path/To/ConfigFile>Command-line Options命令行选项
If you are using command-line options:如果您正在使用命令行选项:
Revert to the original port number.恢复到原始端口号。Include包括--replSet.--replSet。Include如果是分片成员,则包括--shardsvrif a shard member or--configsvrif a config server member.--shardsvr,如果是配置服务器成员,则包含--configsvr。Remove parameter删除参数skipShardingConfigurationChecks.skipShardingConfigurationChecks。Remove parameter删除参数disableLogicalSessionCacheRefresh.disableLogicalSessionCacheRefresh。
For example:例如:
mongod --port 27018 --replSet shardA --shardsvr
Other settings (e.g. 其他设置(例如--dbpath, etc.) remain the same.--dbpath等)保持不变。
Allow replication to catch up on this member.允许复制赶上此成员。
C4. Repeat the Procedure for the Remaining Secondaries for the Shard对分片的剩余Secondary重复该程序
Once the member catches up with the other members of the set, repeat the procedure one member at a time for the remaining secondary members for the shard:一旦成员赶上了集合中的其他成员,就对分片的其余secondary成员一次重复一个成员的过程:
C5. Build the Index on the Primary在Primary建立索引
When all the secondaries for the shard have the new index, step down the primary for the shard, restart it as a standalone using the procedure described above, and build the index on the former primary:当分片的所有次级都有新索引时,关闭分片的primary,使用上述过程将其作为独立的重新启动,并在前一个primary上构建索引:
Use the使用rs.stepDown()method inmongoshto step down the primary. Upon successful stepdown, the current primary becomes a secondary and the replica set members elect a new primary.mongosh中的rs.stepDown()方法来降低主值。成功降级后,当前主服务器将成为次服务器,副本集成员将选择一个新的主服务器。- C1.
Stop One Secondary and Restart as a Standalone停止一个Secondary备并作为单机程序重新启动 - C2.
Build the Index构建索引 - C3.
Restart the Program以副本集成员身份重新启动程序mongodas a Replica Set Membermongod
D. Repeat for the Other Affected Shards对其他受影响的分片重复此操作
Once you finish building the index for a shard, repeat C. Build Indexes on the Shards That Contain Collection Chunks for the other affected shards.一旦你完成了一个分片的索引构建,重复C.为其他受影响的分片在包含集合块的分片上构建索引。
E. Enable Migrations启用迁移
Connect 将mongosh to a mongos instance in the sharded cluster and re-enable the migration with setAllowMigrations:mongosh连接到分片集群中的mongos实例,并使用setAllowMigrations重新启用迁移:
db.adminCommand(
{
setAllowMigrations: "<db>.<collection>",
allowMigrations: true
}
)
If the command returns the following error, it means the collection is unsharded. You can safely ignore the error.如果该命令返回以下错误,则表示该集合未被记录。您可以放心地忽略该错误。
MongoServerError[NamespaceNotSharded]: Collection must be sharded so migrations can be blockedAdditional Information附加信息
A sharded collection has an inconsistent index if the collection does not have the exact same indexes (including the index options) on each shard that contains chunks for the collection. Although inconsistent indexes should not occur during normal operations, inconsistent indexes can occur, such as:如果一个分片集合在包含该集合块的每个分片上没有完全相同的索引(包括索引选项),则该分片集合的索引不一致。虽然在正常操作过程中不应出现不一致的索引,但可能会出现不一致索引,例如:
When a user is creating an index with a当用户创建具有uniquekey constraint and one shard contains a chunk with duplicate documents. In such cases, the create index operation may succeed on the shards without duplicates but not on the shard with duplicates.unique键约束的索引时,一个分片包含一个包含重复文档的块。在这种情况下,创建索引操作可能会在没有重复的分片上成功,但在有重复的分片上则不会成功。When a user is creating an index across the shards in a rolling manner but either fails to build the index for an associated shard or incorrectly builds an index with different specification.当用户以滚动方式在分片之间创建索引,但未能为相关分片构建索引,或者错误地构建了具有不同规范的索引时。
The config server primary periodically checks for index inconsistencies across the shards for sharded collections. 配置服务器primary定期检查分片集合的分片索引是否不一致。To configure these periodic checks, see 要配置这些定期检查,请参阅启用enableShardedIndexConsistencyCheck and shardedIndexConsistencyCheckIntervalMS.enableShardedIndexConsistencyCheck和shardedIndexConsistencyCheckIntervalMS。
The command 命令serverStatus returns the field shardedIndexConsistency to report on index inconsistencies when run on the config server primary.serverStatus返回字段shardedIndexConsistency,以在配置服务器主服务器上运行时报告索引不一致。
To check if a sharded collection has inconsistent indexes, see Find Inconsistent Indexes Across Shards.要检查分片集合是否具有不一致的索引,请参阅跨分片查找不一致索引。