MongoDB MongoDB mongos instances route queries and write operations to shards in a sharded cluster. mongos实例将查询和写入操作路由到分片集群中的分片。从应用程序的角度来看,mongos provides the only interface to a sharded cluster from the perspective of applications. Applications never connect or communicate directly with the shards.mongos提供了分片集群的唯一接口。应用程序从不直接与分片连接或通信。
The mongos tracks what data is on which shard by caching the metadata from the config servers. mongos通过缓存配置服务器的元数据来跟踪哪个分片上的数据。The mongos uses the metadata to route operations from applications and clients to the mongod instances. mongos使用元数据将操作从应用程序和客户端路由到mongod实例。A mongos has no persistent state and consumes minimal system resources.mongos没有持久状态,消耗的系统资源最少。
The most common practice is to run 最常见的做法是在与应用程序服务器相同的系统上运行mongos instances on the same systems as your application servers, but you can maintain mongos instances on the shards or on other dedicated resources. mongos实例,但您可以在分片或其他专用资源上维护mongos示例。See also Number of 另请参见mongos and Distribution.mongos的数量和分布。
Routing And Results Process路线和结果流程
A mongos instance routes a query to a cluster by:mongos实例通过以下方式将查询路由到集群:
Determining the list of shards that must receive the query.确定必须接收查询的分片列表。Establishing a cursor on all targeted shards.在所有目标分片上建立游标。
The 然后,mongos then merges the data from each of the targeted shards and returns the result document. Certain query modifiers, such as sorting, are performed on each shard before mongos retrieves the results.mongos合并来自每个目标分片的数据,并返回结果文档。在mongos检索结果之前,对每个分片执行某些查询修饰符,如排序。
Aggregation operations running on multiple shards may route results back to the 在多个分片上运行的聚合操作可能会将结果路由回mongos to merge results if they don't need to run on the database's primary shard.mongos,以便在不需要在数据库的primary分片上执行的情况下合并结果。
There are two cases in which a pipeline is ineligible to run on 有两种情况下,管道没有资格在mongos.mongos上运行。
The first case occurs when the merge part of the split pipeline contains a stage which must run on a specific shard. For instance, if 第一种情况发生在拆分管道的合并部分包含必须在特定分片上运行的阶段时。例如,如果$lookup requires access to an unsharded collection in the same database as the sharded collection on which the aggregation is running, the merge runs on the shard that hosts the unsharded collection.$lookup需要访问与运行聚合的分片集合位于同一数据库中的未分片集合,则合并将在承载未分片集合的分片上运行。
The second case occurs when the merge part of the split pipeline contains a stage which may write temporary data to disk, such as 第二种情况发生在拆分管道的合并部分包含一个可能将临时数据写入磁盘的阶段(如$group, and the client has specified allowDiskUse:true. $group),并且客户端已指定allowDiskUse:true时。In this case, assuming that there are no other stages in the merge pipeline which require the primary shard, the merge runs on a randomly-selected shard in the set of shards targeted by the aggregation.在这种情况下,假设合并管道中没有其他需要主分片的阶段,合并将在聚合目标分片集中随机选择的一个分片上运行。
For more information on how the work of aggregation is split among components of a sharded cluster query, use 有关如何在分片集群查询的组件之间分割聚合工作的更多信息,请使用explain:true as a parameter to the aggregate() call. The return includes three JSON objects:explain:true作为aggregate()调用的参数。返回包含三个JSON对象:
mergeTypeshows where the stage of the merge happens ("anyShard", "specificShard", or "router").显示合并阶段发生的位置(“anyShard”、“specificShard”或“router”)。When当mergeTypeisspecificShard, the aggregate output includes amergeShardproperty that contains the shard ID of the merging shard.mergeType指定为分片时,聚合输出包括一个mergeShard属性,该属性包含合并分片的分片ID。splitPipelineshows which operations in your pipeline have run on individual shards.显示管道中的哪些操作在单个分片上运行。shardsshows the work each shard has done.显示每个分片所做的工作。
In some cases, when the shard key or a prefix of the shard key is a part of the query, the 在某些情况下,当分片键或分片键的前缀是查询的一部分时,mongos performs a targeted operation, routing queries to a subset of shards in the cluster.mongos会执行定向操作,将查询路由到集群中的分片子集。
mongos performs a broadcast operation for queries that do not include the shard key, routing queries to all shards in the cluster. mongos对不包含分片键的查询执行广播操作,将查询路由到集群中的所有分片。Some queries that do include the shard key may still result in a broadcast operation depending on the distribution of data in the cluster and the selectivity of the query.根据集群中数据的分布和查询的选择性,一些包含分片键的查询仍可能导致广播操作。
See Targeted Operations vs. Broadcast Operations for more on targeted and broadcast operations.有关定向和广播操作的更多信息,请参阅定向操作与广播操作。
How mongos Handles Query Modifiersmongos如何处理查询修饰符
mongos Handles Query ModifiersSorting排序
If the result of the query is not sorted, the 如果查询的结果未排序,mongos instance opens a result cursor that "round robins" results from all cursors on the shards.mongos实例将打开一个结果游标,该游标“舍入robins”是分片上所有游标的结果。
Limits限制
If the query limits the size of the result set using the 如果查询使用limit() cursor method, the mongos instance passes that limit to the shards and then re-applies the limit to the result before returning the result to the client.limit()游标方法限制了结果集的大小,mongos实例会将该限制传递给分片,然后在将结果返回给客户端之前将该限制重新应用于结果。
Skips跳过
If the query specifies a number of records to skip using the 如果查询使用skip() cursor method, the mongos cannot pass the skip to the shards, but rather retrieves unskipped results from the shards and skips the appropriate number of documents when assembling the complete result.skip()游标方法指定要跳过的记录数量,则mongos不能将跳过传递给分片,而是从分片中检索未跳过的结果,并在组装完整结果时跳过适当数量的文档。
When used in conjunction with a 当与limit(), the mongos passes the limit plus the value of the skip() to the shards to improve the efficiency of these operations.limit()结合使用时,mongos会将limit加上skip()的值传递给分片,以提高这些操作的效率。
Read Preference and Shards阅读偏好和分片
For sharded clusters, 对于分片集群,mongos applies the read preference when reading from the shards. mongos在从分片读取时应用读取偏好。The member selected is governed by both the read preference and 所选成员受读取首选项和replication.localPingThresholdMs settings, and is re-evaluated for each operation.replication.localPingThresholdMs设置的控制,并对每个操作进行重新评估。
For details on read preference and sharded clusters, see Read Preference and Shards.有关读取首选项和分片群集的详细信息,请参阅读取首选项与分片。
Confirm Connection to mongos Instances确认与mongos实例的连接
mongos InstancesTo detect if the MongoDB instance that your client is connected to is 要检测客户端连接到的MongoDB实例是否是mongos, use the hello command. mongos,请使用hello命令。When a client connects to a 当客户端连接到mongos, hello returns a document with a msg field that holds the string isdbgrid. For example:mongos时,hello返回一个包含msg字段的文档,该字段包含字符串isdbgrid。例如:
{
"isWritablePrimary" : true,
"msg" : "isdbgrid",
"maxBsonObjectSize" : 16777216,
"ok" : 1,
...
}
If the application is instead connected to a 如果应用程序连接到mongod, the returned document does not include the isdbgrid string.mongod,则返回的文档不包括isdbgrid字符串。
Targeted Operations vs. Broadcast Operations定向操作与广播操作
Generally, the fastest queries in a sharded environment are those that 通常,分片环境中最快的查询是mongos route to a single shard, using the shard key and the cluster meta data from the config server. mongos使用分片键和配置服务器中的集群元数据路由到单个分片的查询。These targeted operations use the shard key value to locate the shard or subset of shards that satisfy the query document.这些定向操作使用分片键值来定位满足查询文档的分片或分片子集。
For queries that don't include the shard key, 对于不包含分片键的查询,mongos must query all shards, wait for their responses and then return the result to the application. mongos必须查询所有分片,等待它们的响应,然后将结果返回给应用程序。These "scatter/gather" queries can be long running operations.这些“分散/聚集”查询可能是长时间运行的操作。
Broadcast Operations广播操作
mongos instances broadcast queries to all shards for the collection unless the mongos can determine which shard or subset of shards stores this data.mongos实例将查询广播到集合的所有分片,除非mongos可以确定哪个分片或分片子集存储了这些数据。
After the mongos receives responses from all shards, it merges the data and returns the result document. mongos收到所有分片的响应后,会合并数据并返回结果文档。The performance of a broadcast operation depends on the overall load of the cluster, as well as variables like network latency, individual shard load, and number of documents returned per shard. 广播操作的性能取决于集群的整体负载,以及网络延迟、单个分片负载和每个分片返回的文档数量等变量。Whenever possible, favor operations that result in targeted operation over those that result in a broadcast operation.只要有可能,优先选择导致定向操作的操作,而不是导致广播操作的操作。
Multi-update operations are always broadcast operations.多更新操作始终是广播操作。
The 除非查询文档完整地指定了分片键,否则updateMany() and deleteMany() methods are broadcast operations, unless the query document specifies the shard key in full.updateMany()和deleteMany()方法都是广播操作。
Targeted Operations定向操作
mongos can route queries that include the shard key or the prefix of a compound shard key a specific shard or set of shards. 可以将包含分片键或复合分片键前缀的查询路由到特定的分片或一组分片。mongos uses the shard key value to locate the chunk whose range includes the shard key value and directs the query at the shard containing that chunk.使用分片键值来定位其范围包括分片键值的块,并将查询定向到包含该块的分片。
For example, if the shard key is:例如,如果分片键是:
{ a: 1, b: 1, c: 1 }
The mongos program can route queries that include the full shard key or either of the following shard key prefixes at a specific shard or set of shards:mongos程序可以在特定分片或一组分片上路由包含完整分片键或以下任一分片键前缀的查询:
{ a: 1 }
{ a: 1, b: 1 }
All 所有insertOne() operations target to one shard. insertOne()操作都指向一个分片。Each document in the insertMany() array targets to a single shard, but there is no guarantee all documents in the array insert into a single shard.insertMany()数组中的每个文档都指向单个分片,但不能保证数组中的所有文档都插入到单个分片中。
All 所有updateOne(), replaceOne() and deleteOne() operations must include the shard key or _id in the query document. updateOne()、replaceOne()和deleteOne()操作都必须在查询文档中包含分片键或_id。MongoDB returns an error if these methods are used without the shard key or 如果在没有分片键或_id._id的情况下使用这些方法,MongoDB将返回错误。
Depending on the distribution of data in the cluster and the selectivity of the query, 根据集群中数据的分布和查询的选择性,mongos may still perform a broadcast operation to fulfill these queries.mongos仍然可以执行广播操作来完成这些查询。
Index Use索引使用
When a shard receives a query, it uses the most efficient index available to fulfill that query. The index used may be either the shard key index or another eligible index present on the shard.当一个分片收到一个查询时,它会使用最有效的索引来完成该查询。使用的索引可以是分片键索引或分片上存在的另一个符合条件的索引。
Sharded Cluster Security分片集群安全
Use Self-Managed Internal/Membership Authentication to enforce intra-cluster security and prevent unauthorized cluster components from accessing the cluster. 使用自我管理的内部/成员身份验证来强制实施集群内安全,并防止未经授权的集群组件访问集群。You must start each 您必须使用适当的安全设置启动集群中的每个mongod or mongos in the cluster with the appropriate security settings in order to enforce internal authentication.mongod或mongos,以强制执行内部身份验证。
Starting in MongoDB 5.3, SCRAM-SHA-1 cannot be used for intra-cluster authentication. Only SCRAM-SHA-256 is supported.从MongoDB 5.3开始,SCRAM-SHA-1不能用于集群内身份验证。仅支持SCRAM-SHA-256。
In previous MongoDB versions, SCRAM-SHA-1 and SCRAM-SHA-256 can both be used for intra-cluster authentication, even if SCRAM is not explicitly enabled.在以前的MongoDB版本中,即使没有明确启用SCRAM,SCRAM-SHA-1和SCRAM-SHA-256都可以用于集群内身份验证。
See Deploy Self-Managed Sharded Cluster with Keyfile Authentication for a tutorial on deploying a secured sharded cluster.有关部署安全分片集群的教程,请参阅部署具有键文件身份验证的自我管理分片集群。
Cluster Users群集用户
Sharded clusters support Role-Based Access Control in Self-Managed Deployments (RBAC) for restricting unauthorized access to cluster data and operations. 分片集群支持自管理部署(RBAC)中的基于角色的访问控制,以限制对集群数据和操作的未经授权的访问。You must start each 为了实施RBAC,您必须使用mongod in the cluster, including the config servers, with the --auth option in order to enforce RBAC. --auth选项启动集群中的每个mongod,包括配置服务器。Alternatively, enforcing Self-Managed Internal/Membership Authentication for inter-cluster security also enables user access controls via RBAC.或者,为集群间安全实施自我管理内部/成员身份验证也可以通过RBAC实现用户访问控制。
With RBAC enforced, clients must specify a 在实施RBAC的情况下,客户端在连接到--username, --password, and --authenticationDatabase when connecting to the mongos in order to access cluster resources.mongos时必须指定--username、--password和--authenticationDatabase才能访问集群资源。
Each cluster has its own cluster users. These users cannot be used to access individual shards.每个集群都有自己的集群用户。这些用户不能用于访问单个分片。
See Enable Access Control on Self-Managed Deployments for a tutorial on enabling adding users to an RBAC-enabled MongoDB deployment.有关启用向启用RBAC的MongoDB部署添加用户的教程,请参阅在自我管理部署上启用访问控制。
Metadata Operations元数据操作
mongos uses 对影响分片集群元数据的以下操作使用"majority" write concern for the following operations that affect the sharded cluster metadata:"majority"写入关注:
Additional Information附加信息
FCV CompatibilityFCV兼容性
The mongos binary cannot connect to mongod instances whose feature compatibility version (FCV) is greater than that of the mongos. mongos二进制文件无法连接到功能兼容性版本(FCV)大于mongos的mongod实例。For example, you cannot connect a MongoDB 4.0 version 例如,您无法将MongoDB 4.0版本的mongos to a 4.2 sharded cluster with FCV set to 4.2. mongos连接到FCV设置为4.2的4.2分片集群。You can, however, connect a MongoDB 4.0 version 但是,您可以将MongoDB 4.0版本的mongos to a 4.2 sharded cluster with FCV set to 4.0.mongos连接到FCV设置为4.0的4.2分片集群。
Full Time Diagnostic Data Capture Requirements全职诊断数据采集要求
mongod includes a Full Time Diagnostic Data Capture mechanism to assist MongoDB engineers with troubleshooting deployments. 包括一个全职诊断数据捕获机制,以协助MongoDB工程师对部署进行故障排除。If this thread fails, it terminates the originating process. 如果此线程失败,它将终止发起进程。To avoid the most common failures, confirm that the user running the process has permissions to create the FTDC 为了避免最常见的故障,请确认运行该流程的用户有权创建FTDCdiagnostic.data directory. diagnostic.data目录。For 对于mongod the directory is within storage.dbPath. mongod,目录位于storage.dbPath中。For 对于mongos it is parallel to systemLog.path.mongos来说,它与systemLog.path并行。
Connection Pools连接池
Starting in MongoDB 4.2, MongoDB adds the parameter 从MongoDB 4.2开始,MongoDB添加了参数ShardingTaskExecutorPoolReplicaSetMatching. ShardingTaskExecutorPoolReplicaSetMatching。This parameter determines the minimum size of the 此参数确定mongod / mongos instance's connection pool to each member of the sharded cluster. This value can vary during runtime.mongod/mongos实例到分片集群每个成员的连接池的最小大小。此值在运行时可能会有所不同。
mongod and mongos maintain connection pools to each replica set secondary for every replica set in the sharded cluster. mongod和mongos为分片集群中的每个副本集维护连接池。By default, these pools have a number of connections that is at least the number of connections to the primary.默认情况下,这些池的连接数量至少与主池的连接数相等。
To modify, see 要修改,请参阅ShardingTaskExecutorPoolReplicaSetMatching.ShardingTaskExecutorPoolReplicaSetMatching。
Using Aggregation Pipelines with Clusters在集群中使用聚合管道
For more information on how sharding works with aggregations, read the sharding chapter in the Practical MongoDB Aggregations e-book.有关分片如何与聚合一起工作的更多信息,请阅读《实用MongoDB聚合》电子书中的分片章节。