Docs HomeMongoDB Manual

$changeStreamSplitLargeEvent (aggregation)

On this page本页内容

Definition定义

$changeStreamSplitLargeEvent

New in version 7.0. 7.0版新增。

If a change stream has large events that exceed 16 MB, a BSONObjectTooLarge exception is returned. 如果变更流包含超过16 MB的大型事件,则返回BSONObjectTooLarge异常。Starting in MongoDB 7.0, you can use a $changeStreamSplitLargeEvent stage to split the events into smaller fragments.从MongoDB 7.0开始,您可以使用$changeStreamSplitTargetEvent阶段将事件拆分为更小的片段。

You should only use $changeStreamSplitLargeEvent when strictly necessary. For example, if your application requires full document pre- or post-images, and generates large events that exceed 16 MB, use $changeStreamSplitLargeEvent.只有在绝对必要的情况下,才应使用$changeStreamSplitTargetEvent。例如,如果您的应用程序需要完整的文档前映像或后映像,并且生成的大型事件超过16 MB,请使用$changeStreamSplitTargetEvent

Before you decide to use $changeStreamSplitLargeEvent, you should first try to reduce the change event size. 在决定使用$changeStreamSplitTargetEvent之前,应首先尝试减小更改事件的大小。For example:例如:

  • Don't request document pre- or post-images unless your application requires them. 除非您的应用程序需要,否则不要要求文档前或后图像。This generates fullDocument and fullDocumentBeforeChange fields in more cases, which are typically the largest objects in a change event.在更多情况下,这会生成fullDocumentfullDocumentBeforeChange字段,它们通常是更改事件中最大的对象。
  • Use a $project stage to include only the fields necessary for your application. 使用$project阶段仅包括应用程序所需的字段。This reduces the change event size and avoids the additional time to split large events into fragments. This allows more change events to be returned in each batch.这减少了更改事件的大小,并避免了将大型事件拆分为片段的额外时间。这允许在每个批次中返回更多的更改事件。

You can only have one $changeStreamSplitLargeEvent stage in your pipeline, and it must be the last stage. 管道中只能有一个$changeStreamSplitTargetEvent阶段,并且它必须是最后一个阶段。You can only use $changeStreamSplitLargeEvent in a $changeStream pipeline.只能在$changeStream管道中使用$changeStreamSplitTargetEvent

$changeStreamSplitLargeEvent syntax:语法:

{
$changeStreamSplitLargeEvent: {}
}

Behavior行为

$changeStreamSplitLargeEvent splits events that exceed 16 MB into fragments and returns the fragments sequentially using the change stream cursor.将超过16MB的事件拆分为片段,并使用更改流游标按顺序返回片段。

The fragments are split so that the maximum number of fields are returned in the first fragment. 片段被分割,使得在第一个片段中返回最大数量的字段。This ensures the event context is returned as quickly as possible.这样可以确保事件上下文尽快返回。

When the change event is split, only the size of top-level fields are used. $changeStreamSplitLargeEvent does not recursively process or split subdocuments. 拆分更改事件时,只使用顶级字段的大小$changeStreamSplitTargetEvent不递归处理或拆分子文档。For example, if you use a $project stage to create a change event with a single field that is 20 MB in size, the event is not split and the stage returns an error.例如,如果使用$project阶段创建一个具有20 MB大小的单个字段的更改事件,则不会拆分该事件,并且该阶段将返回错误。

Each fragment has a resume token. 每个片段都有一个恢复标记。A stream that is resumed using a fragment's token will either:使用片段的令牌恢复的流将:

  • Begin a new stream from the subsequent fragment.从后续片段开始一个新的流。
  • Start at the next event if resuming from the final fragment in the sequence.如果从序列中的最后一个片段恢复,则从下一个事件开始。

Each fragment for an event includes a splitEvent document:事件的每个片段都包括一个splitEvent文档:

splitEvent: {
fragment: <int>,
of: <int>
}

The following table describes the fields.下表介绍了这些字段。

Field字段Description描述
fragmentFragment index, starting at 1.片段索引,从1开始。
ofTotal number of fragments for the event.事件的分片总数。

Example实例

The example scenario in this section shows the use of $changeStreamSplitLargeEvent with a new collection named myCollection.本节中的示例场景显示了对名为myCollection的新集合使用$changeStreamSplitTargetEvent

Create myCollection and insert one document with just under 16 MB of data:创建myCollection并插入一个数据不到16 MB的文档:

db.myCollection.insertOne(
{ _id: 0, largeField: "a".repeat( 16 * 1024 * 1024 - 1024 ) }
)

largeField contains the repeated letter a.包含重复的字母a

Enable changeStreamPreAndPostImages for myCollection, which allows a change stream to retrieve a document as it was before an update (pre-image) and after an update (post-image):myCollection启用changeStreamPreAndPostImages,这允许更改流检索更新前(前图像)和更新后(后图像)的文档:

db.runCommand( {
collMod: "myCollection",
changeStreamPreAndPostImages: { enabled: true }
} )

Create a change stream cursor to monitor changes to myCollection using db.collection.watch():使用db.collection.watch()创建一个更改流游标来监视对myCollection的更改:

myChangeStreamCursor = db.myCollection.watch(
[ { $changeStreamSplitLargeEvent: {} } ],
{ fullDocument: "required", fullDocumentBeforeChange: "required" }
)

For the change stream event:对于变更流事件:

  • fullDocument: "required" includes the document post-image.包括文档张贴图像。
  • fullDocumentBeforeChange: "required" includes the document pre-image.包括文档预图像。

For details, see $changeStream.有关详细信息,请参阅$changeStream

Update the document in myCollection, which also produces a change stream event with the document pre- and post-images:更新myCollection中的文档,它还会生成一个带有文档前图像和后图像的更改流事件:

db.myCollection.updateOne(
{ _id: 0 },
{ $set: { largeField: "b".repeat( 16 * 1024 * 1024 - 1024 ) } }
)

largeField now contains the repeated letter b.现在包含重复的字母b

Retrieve the fragments from myChangeStreamCursor using the next() method and store the fragments in objects named firstFragment, secondFragment, and thirdFragment:使用next()方法从myChangeStreamCursor中检索片段,并将片段存储在名为firstFragmentsecondFragmentthirdFragment的对象中:

const firstFragment = myChangeStreamCursor.next()
const secondFragment = myChangeStreamCursor.next()
const thirdFragment = myChangeStreamCursor.next()

Show firstFragment.splitEvent:显示firstFragment.splitEvent

firstFragment.splitEvent

Output with the fragment details:输出片段详细信息:

splitEvent: { fragment: 1, of: 3 }

Similarly, secondFragment.splitEvent and thirdFragment.splitEvent return:类似地,secondFragment.splitEventthirdFragment.splitEvent返回:

splitEvent: { fragment: 2, of: 3 }
splitEvent: { fragment: 3, of: 3 }

To examine the object keys for firstFragment:要检查firstFragment的对象键,请执行以下操作:

Object.keys( firstFragment )

Output:输出:

[
'_id',
'splitEvent',
'wallTime',
'clusterTime',
'operationType',
'documentKey',
'ns',
'fullDocument'
]

To examine the size in bytes for firstFragment.fullDocument:要检查firstFragment.fullDocument的大小(以字节为单位),请执行以下操作:

bsonsize( firstFragment.fullDocument )

Output:输出:

16776223

secondFragment contains the fullDocumentBeforeChange pre-image, which is approximately 16 MB in size. secondFragment包含完整的DocumentBeforeChange预映像,其大小约为16 MB。The following example shows the object keys for secondFragment:以下示例显示了secondFragment的对象键:

Object.keys( secondFragment )

Output:输出:

[ '_id', 'splitEvent', 'fullDocumentBeforeChange' ]

thirdFragment contains the updateDescription field, which is approximately 16 MB in size. The following example shows the object keys for thirdFragment:thirdFragment包含updateDescription字段,该字段的大小约为16MB。以下示例显示了thirdFragment的对象键:

Object.keys( thirdFragment )

Output:输出:

[ '_id', 'splitEvent', 'updateDescription' ]

For more information about change streams and events, see Change Events.有关更改流和事件的更多信息,请参阅更改事件