Database Manual / Sharding / Shard Keys

Choose a Shard Key选择分片键

The choice of shard key affects the creation and distribution of chunks across the available shards. The distribution of data affects the efficiency and performance of operations within the sharded cluster.分片键的选择会影响块在可用分片上的创建和分布。数据的分布会影响分片集群内操作的效率和性能。

The ideal shard key allows MongoDB to distribute documents evenly throughout the cluster while also facilitating common query patterns.理想的分片键允许MongoDB在整个集群中均匀分布文档,同时也促进常见的查询模式。

When you choose your shard key, consider:当您选择分片键时,请考虑:

Note

Important

If you regularly change a document's shard key value so that the value is in a shard key range owned by a different shard, it may impact cluster performance due to the additional resources involved in migrating the document between shards. 如果您定期更改文档的分片键值,使该值位于不同分片拥有的分片键范围内,则由于在分片之间迁移文档所涉及的额外资源,可能会影响集群性能。For details, see Data Partitioning with Chunks and db.collection.updateOne().有关详细信息,请参阅使用块进行数据分区db.collection.updateOne()

Shard Key Cardinality分片键基数

The cardinality of a shard key determines the maximum number of chunks the balancer can create. Where possible, choose a shard key with high cardinality. A shard key with low cardinality reduces the effectiveness of horizontal scaling in the cluster.分片键的基数决定了平衡器可以创建的最大块数。在可能的情况下,选择具有高基数的分片键。基数较低的分片键会降低集群中水平扩展的有效性。

Each unique shard key value can exist on no more than a single chunk at any given time. Consider a dataset that contains user data with a continent field. 在任何给定时间,每个唯一的分片键值只能存在于一个块上。考虑一个包含用户数据和continent(大陆)字段的数据集。If you chose to shard on continent, the shard key would have a cardinality of 7. 如果你选择在continent上分片,分片键的基数将为7A cardinality of 7 means there can be no more than 7 chunks within the sharded cluster, each storing one unique shard key value. 基数为7意味着分片集群内最多只能有7个块,每个块存储一个唯一的分片键值。This constrains the number of effective shards in the cluster to 7 as well - adding more than seven shards would not provide any benefit.这也将集群中有效分片的数量限制在7个——添加超过7个分片不会带来任何好处。

The following image illustrates a sharded cluster using the field X as the shard key. If X has low cardinality, the distribution of inserts may look similar to the following:下图显示了使用字段X作为分片键的分片集群。如果X的基数较低,插入的分布可能类似于以下内容:

Diagram of poor shard key distribution due to low cardinality

If your data model requires sharding on a key that has low cardinality, consider using an indexed compound of fields to increase cardinality.如果数据模型需要对基数较低的键进行分片,请考虑使用字段的索引复合来增加基数。

A shard key with high cardinality does not, on its own, guarantee even distribution of data across the sharded cluster. 具有高基数的分片键本身并不能保证数据在分片集群中的均匀分布。The frequency of the shard key and the potential for monotonically changing shard key values also contribute to the distribution of the data.分片键的频率分片键值单调变化的可能性也有助于数据的分布。

Shard Key Frequency分片键频率

The frequency of the shard key represents how often a given shard key value occurs in the data. 分片键的frequency表示给定分片键值在数据中出现的频率。If the majority of documents contain only a subset of the possible shard key values, then the chunks storing the documents with those values can become a bottleneck within the cluster. 如果大多数文档只包含可能的分片键值的一个子集,那么存储具有这些值的文档的块可能会成为集群内的瓶颈。Furthermore, as those chunks grow, they may become indivisible chunks as they cannot be split any further. 此外,随着这些块的增长,它们可能会成为不可分割的块,因为它们不能再被分割。This reduces the effectiveness of horizontal scaling within the cluster.这降低了集群内水平扩展的有效性。

The following image illustrates a sharded cluster using the field X as the shard key. If a subset of values for X occur with high frequency, the distribution of inserts may look similar to the following:下图显示了使用字段X作为分片键的分片集群。如果X的一个子集频繁出现,插入的分布可能类似于以下内容:

Diagram of poor shard key distribution due to high frequency

If your data model requires sharding on a key that has high frequency values, consider using a compound index using a unique or low frequency value.如果数据模型需要对具有高频值的键进行分片,请考虑使用使用唯一或低频值的复合索引

A shard key with low frequency does not, on its own, guarantee even distribution of data across the sharded cluster. 低频率的分片键本身并不能保证数据在分片集群中的均匀分布。The cardinality of the shard key and the potential for monotonically changing shard key values also contribute to the distribution of the data.分片键的基数分片键值单调变化的可能性也有助于数据的分布。

Monotonically Changing Shard Keys单调变化的分片键

A shard key on a value that increases or decreases monotonically is more likely to distribute inserts to a single chunk within the cluster.单调递增或递减的值上的分片键更有可能将插入分布到集群内的单个块中。

This occurs because every cluster has a chunk that captures a range with an upper bound of MaxKey. 这是因为每个集群都有一个块,它捕获了一个上限为MaxKey的范围。maxKey always compares as higher than all other values. maxKey总是比其他所有值都高。Similarly, there is a chunk that captures a range with a lower bound of MinKey. 同样,有一个块捕获下限为MinKey的范围。minKey always compares as lower than all other values.minKey总是比其他所有值都低。

If the shard key value is always increasing, all new inserts are routed to the chunk with maxKey as the upper bound. 如果分片键值总是在增加,则所有新插入都会被路由到以maxKey为上限的块。If the shard key value is always decreasing, all new inserts are routed to the chunk with minKey as the lower bound. 如果分片键值总是递减,则所有新插入都将路由到以minKey为下限的块。The shard containing that chunk becomes the bottleneck for write operations.包含该块的分片成为写操作的瓶颈。

To optimize data distribution, the chunks that contain the global maxKey (or minKey) do not stay on the same shard. 为了优化数据分布,包含全局maxKey(或minKey)的块不会留在同一个分片上。When a chunk is split, the new chunk with the maxKey (or minKey) chunk is located on a different shard.当一个块被分割时,具有maxKey(或minKey)块的新块位于不同的分片上。

The following image illustrates a sharded cluster using the field X as the shard key. If the values for X are monotonically increasing, the distribution of inserts may look similar to the following:下图显示了使用字段X作为分片键的分片集群。如果X的值单调递增,则插入的分布可能类似于以下内容:

Diagram of poor shard key distribution due to monotonically increasing or decreasing shard key

If the shard key value was monotonically decreasing, then all inserts would route to Chunk A instead.如果分片键值单调递减,那么所有插入都将路由到Chunk A

If your data model requires sharding on a key that changes monotonically, consider using Hashed Sharding.如果数据模型需要对单调变化的键进行分片,请考虑使用哈希分片

A shard key that does not change monotonically does not, on its own, guarantee even distribution of data across the sharded cluster. 不单调变化的分片键本身并不能保证数据在分片集群中的均匀分布。The cardinality and frequency of the shard key also contribute to the distribution of the data.分片键的基数频率也有助于数据的分布。

Sharding Query Patterns分片查询模式

The ideal shard key distributes data evenly across the sharded cluster while also facilitating common query patterns. When you choose a shard key, consider your most common query patterns and whether a given shard key covers them.理想的分片键在分片集群中均匀分布数据,同时也促进了常见的查询模式。当你选择一个分片键时,考虑你最常见的查询模式,以及给定的分片键是否覆盖了它们。

In a sharded cluster, the mongos routes queries to only the shards that contain the relevant data if the queries contain the shard key. 在分片集群中,如果查询包含分片键,mongos只会将查询路由到包含相关数据的分片。When the queries do not contain the shard key, the queries are broadcast to all shards for evaluation. 当查询不包含分片键时,查询会广播到所有分片进行评估。These types of queries are called scatter-gather queries. Queries that involve multiple shards for each request are less efficient and do not scale linearly when more shards are added to the cluster.这些类型的查询称为分散-聚集查询。每个请求涉及多个分片的查询效率较低,当向集群中添加更多分片时,查询不会线性扩展。

This does not apply for aggregation queries that operate on a large amount of data. In these cases, scatter-gather can be a useful approach that allows the query to run in parallel on all shards.这不适用于对大量数据进行操作的聚合查询。在这些情况下,分散集合是一种有用的方法,它允许查询在所有分片上并行运行。

Use Shard Key Analyzer in 7.0 to Find Your Shard Key使用7.0中的分片键分析器查找分片键

Starting in 7.0, MongoDB makes it easier to choose your shard key. You can use analyzeShardKey which calculates metrics for evaluating a shard key for an unsharded or sharded collection. 从7.0开始,MongoDB使选择分片键变得更加容易。您可以使用analyzeShardKey来计算用于评估未分片或分片集合的分片键的指标。Metrics are based on sampled queries, allowing you to make a data-driven choice for your shard key.度量基于采样查询,允许您为分片键做出数据驱动的选择。

Enable Query Sampling启用查询采样

To analyze a shard key, you must enable query sampling on the target collection. For more information, see:要分析分片键,必须在目标集合上启用查询采样。有关更多信息,请参阅:

To monitor the query sampling process, use the $currentOp stage. 要监视查询采样过程,请使用$currentOp阶段。For an example, see Sampled Queries.有关示例,请参阅采样查询

Shard Key Analysis Commands分片化键分析命令

To analyze a shard key, see:要分析分片键,请参阅:

analyzeShardKey returns metrics about key characteristics of a shard key and its read and write distribution. The metrics are based on sampled queries.返回关于分片键的关键特性及其读写分布的度量。这些指标基于采样查询。

  • The keyCharacteristics field contains metrics about the cardinality, frequency, and monotonicity of the shard key.keyCharacteristics字段包含有关分片键的基数频率单调性的度量。
  • The readWriteDistribution field contains metrics about the query routing patterns and the load distribution of shard key ranges.readWriteDistribution字段包含有关查询路由模式和分片键范围负载分布的指标