Troubleshoot Sharded Clusters分片群集故障排除

Before You Begin开始之前

Starting in MongoDB 8.0, you can use the directShardOperations role to perform maintenance operations that require you to execute commands directly against a shard.从MongoDB 8.0开始，您可以使用directShardOperations角色执行维护操作，这些操作要求您直接对分片执行命令。

Application Servers or mongos Instances Become Unavailable应用程序服务器或mongos实例变得不可用

If each application server has its own mongos instance, other application servers can continue to access the database. 如果每个应用服务器都有自己的mongos实例，其他应用服务器可以继续访问数据库。Furthermore, mongos instances do not maintain persistent state, and they can restart and become unavailable without losing any state or data. 此外，mongos实例不维护持久状态，它们可以重新启动并变得不可用，而不会丢失任何状态或数据。When a mongos instance starts, it retrieves a copy of the config database and can begin routing queries.当mongos实例启动时，它会检索config数据库的副本，并可以开始路由查询。

A Single Member Becomes Unavailable in a Shard Replica Set分片副本集中的单个成员变得不可用

Replica sets provide high availability for shards. 副本集为分片提供了高可用性。If the unavailable mongod is a primary, then the replica set will elect a new primary. 如果不可用的mongod是primary服务器，则副本集将选择一个新的主服务器。If the unavailable mongod is a secondary, and it disconnects the primary and secondary will continue to hold all data. 如果不可用的mongod是secondary，并且它断开了连接，则主要和次要将继续保存所有数据。In a three member replica set, even if a single member of the set experiences catastrophic failure, two other members have full copies of the data. 在一个由三个成员组成的副本集中，即使其中一个成员发生灾难性故障，其他两个成员也有完整的数据副本。[1]

Always investigate availability interruptions and failures. If a system is unrecoverable, replace it and create a new member of the replica set as soon as possible to replace the lost redundancy.始终调查可用性中断和故障。如果系统不可恢复，请更换它，并尽快创建副本集的新成员，以替换丢失的冗余。

All Members of a Shard Become Unavailable分片的所有成员都不可用

In a sharded cluster, mongod and mongos instances monitor the replica sets in the sharded cluster (e.g. shard replica sets, config server replica set).在分片集群中，mongod和mongos实例监控分片集群内的副本集（例如，分片副本集、配置服务器副本集）。

If all members of a replica set shard are unavailable, all data held in that shard is unavailable. However, the data on all other shards will remain available, and it is possible to read and write data to the other shards. 如果副本集分片的所有成员都不可用，则该分片中保存的所有数据都不可用。但是，所有其他分片上的数据将保持可用，并且可以向其他分片读写数据。However, your application must be able to deal with partial results, and you should investigate the cause of the interruption and attempt to recover the shard as soon as possible.但是，应用程序必须能够处理部分结果，您应该调查中断的原因，并尽快尝试恢复分片。

A Config Server Replica Set Member Become Unavailable配置服务器副本集成员变得不可用

Replica sets provide high availability for the config servers. If an unavailable config server is a primary, then the replica set will elect a new primary.副本集为配置服务器提供了高可用性。如果不可用的配置服务器是primary服务器，则副本集将 选择一个新的主服务器。

If the replica set config server loses its primary and cannot elect a primary, the cluster's metadata becomes read only. 如果副本集配置服务器丢失了其主服务器并且无法选择主服务器，则集群的元数据将变为只读。You can still read and write data from the shards, but no chunk migration or chunk splits will occur until a primary is available.您仍然可以从分片读取和写入数据，但在主分片可用之前，不会发生块迁移或块拆分。

Cursor Fails Because of Stale Config Data由于配置数据过时，游标失败

A query returns the following warning when one or more of the mongos instances has not yet updated its cache of the cluster's metadata from the config database:当一个或多个mongos实例尚未从config数据库更新其集群元数据缓存时，查询会返回以下警告：

could not initialize cursor across all shards because : stale config detected

This warning should not propagate back to your application. 此警告不应传播回应用程序。The warning will repeat until all the mongos instances refresh their caches. 警告将重复，直到所有mongos实例刷新其缓存。To force an instance to refresh its cache, run the flushRouterConfig command.要强制实例刷新其缓存，请运行flushRouterConfig命令。

Shard Keys分片钥匙

To troubleshoot a shard key, see Troubleshoot Shard Keys.要对分片键进行故障排除，请参阅分片键故障排除。

Cluster Availabilit群集可用性y

To ensure cluster availability:为确保集群可用性：

• Each shard should be a replica set, if a specific mongod instance fails, the replica set members will elect another to be primary and continue operation. 每个分片都应该是一个副本集，如果特定的mongod实例发生故障，副本集成员将选择另一个作为primary并继续操作。However, if an entire shard is unreachable or fails for some reason, that data will be unavailable.但是，如果整个分片无法访问或因某种原因发生故障，则该数据将不可用。
• The shard key should allow the mongos to isolate most operations to a single shard. 分片键应该允许mongos将大多数操作隔离到一个分片中。If operations can be processed by a single shard, the failure of a single shard will only render some data unavailable. 如果操作可以由单个分片处理，那么单个分片的失败只会使一些数据不可用。If operations need to access all shards for queries, the failure of a single shard will render the entire cluster unavailable.如果操作需要访问所有分片进行查询，那么单个分片的失败将导致整个集群不可用。

Config Database String Error配置数据库字符串错误

Config servers must be deployed as replica sets. 配置服务器必须作为副本集部署。The mongos instances for the sharded cluster must specify the same config server replica set name but can specify hostname and port of different members of the replica set.分片集群的mongos实例必须指定相同的配置服务器副本集名称，但可以指定副本集不同成员的主机名和端口。

With earlier versions of MongoDB sharded clusters that use the topology of three mirrored mongod instances for config servers, mongos instances in a sharded cluster must specify identical configDB string.对于早期版本的MongoDB分片集群，它们使用三个镜像mongod实例的拓扑结构作为配置服务器，分片集群中的mongos实例必须指定相同的configDB字符串。

Avoid Downtime when Moving Config Servers移动配置服务器时避免停机

Use CNAMEs to identify your config servers to the cluster so that you can rename and renumber your config servers without downtime.使用CNAME在集群中标识配置服务器，以便您可以在不停机的情况下重命名和重新编号配置服务器。

moveChunk commit failed Error

At the end of a chunk migration, the shard must connect to the config database to update the chunk's record in the cluster metadata. 在块迁移结束时，分片必须连接到config数据库，以更新集群元数据中块的记录。If the shard fails to connect to the config database, MongoDB reports the following error:如果分片无法连接到config数据库，MongoDB会报告以下错误：

ERROR: moveChunk commit failed: version is at <n>|<nn> instead of
<N>|<NN>" and "ERROR: TERMINATING"

When this happens, the primary member of the shard's replica set then terminates to protect data consistency. 当这种情况发生时，分片副本集的primary成员将终止以保护数据一致性。If a secondary member can access the config database, data on the shard becomes accessible again after an election.如果secondary成员可以访问配置数据库，那么在选举后，分片上的数据将再次可访问。

The user will need to resolve the chunk migration failure independently. If you encounter this issue, ask the MongoDB Community or MongoDB Support to address this issue.用户需要独立解决块迁移失败。如果您遇到此问题，请向MongoDB社区或MongoDB支持部门咨询以解决此问题。

Inconsistent Sharding Metadata分片元数据不一致

Starting in MongoDB 7.0, the checkMetadataConsistency command is available to check sharding metadata for inconsistencies and corruptions due to bugs in previous releases of MongoDB.从MongoDB 7.0开始，checkMetadataConsistency命令可用于检查分片元数据是否因MongoDB早期版本中的错误而出现不一致和损坏。

Inconsistencies in sharding metadata can originate in cases such as:分片元数据中的不一致可能源于以下情况：

• Clusters upgraded from a pre-5.0 release of MongoDB that may have corrupted data from past DDL operations.从MongoDB 5.0之前版本升级的集群可能已经损坏了过去DDL操作的数据。
• Manual interventions, such as manipulating the Config Database or bypassing mongos to write directly to a shard手动干预，例如操纵配置数据库或绕过mongos直接写入分片.
• Maintenance operations, such as upgrade or downgrade procedures.维护操作，如升级或降级程序。

These inconsistencies can result in incorrect query results or data loss.这些不一致可能会导致不正确的查询结果或数据丢失。

To check sharding metadata for inconsistencies, run the checkMetadataConsistency command:要检查分片元数据是否不一致，请运行checkMetadataConsistency命令：

db.runCommand( { checkMetadataConsistency: 1 } )

{
   cursor: {
      id: Long("0"),
      ns: "test.$cmd.aggregate",
      firstBatch: [
         {
            type: "MisplacedCollection",
            description: "Unsharded collection found on shard different from database primary shard",
            details: {
               namespace: "test.authors",
               shard: "shard02",
               localUUID: new UUID("1ad56770-61e2-48e9-83c6-8ecefe73cfc4")
            }
         }
      ],
   },
   ok: 1
}

Documents returned by the checkMetadataConsistency command indicate the inconsistencies identified by MongoDB in the sharding metadata of the cluster.checkMetadataConsistency命令返回的文档表示MongoDB在集群的分片元数据中识别出的不一致。

For information on inconsistency documents returned by the checkMetadataConsistency command, see Inconsistency Types.有关checkMetadataConsistency命令返回的不一致文档的信息，请参阅不一致类型。