Database Manual / Indexes / Properties

TTL Indexes索引

Note

If you are removing documents to save on storage costs, consider Online Archive in MongoDB Atlas. 如果您要删除文档以节省存储成本,请考虑MongoDB Atlas中的在线存档Online Archive automatically archives infrequently accessed data to fully-managed S3 buckets for cost-effective data tiering.Online Archive会自动将不常访问的数据归档到完全受管理的S3桶中,以实现经济高效的数据分层。

"Time-to-live" (TTL)“生存时间”(TTL) indexes are special single-field indexes that MongoDB can use to automatically remove documents from a collection after a certain amount of time or at a specific clock time. Data expiration is useful for certain types of information like machine generated event data, logs, and session information that only need to persist in a database for a finite amount of time.索引是特殊的单字段索引,MongoDB可以使用它在一定时间或特定时钟时间后自动从集合中删除文档。数据过期对于某些类型的信息很有用,如机器生成的事件数据、日志和会话信息,这些信息只需要在数据库中保留有限的时间。

You can create and manage TTL indexes in the UI for deployments hosted in MongoDB Atlas.您可以在MongoDB Atlas中托管的部署的UI中创建和管理TTL索引

Create a TTL Index创建TTL索引

Warning

After you create a TTL index, it might have a very large number of qualifying documents to delete at once. This large workload might cause performance issues on the server. To avoid these issues, plan to create the index during off hours, or delete qualifying documents in batches before you create the index for future documents.创建TTL索引后,它可能会一次删除大量符合条件的文档。如此大的工作负载可能会导致服务器上的性能问题。为了避免这些问题,请计划在非工作时间创建索引,或者在为未来的文档创建索引之前批量删除符合条件的文档。

To create a TTL index, use createIndex(). Specify an index field that is either a date type or an array that contains date type values. Use the expireAfterSeconds option to specify a TTL value in seconds.要创建TTL索引,请使用createIndex()。指定一个索引字段,该字段可以是日期类型,也可以是包含日期类型值的数组。使用expireAfterSeconds选项以秒为单位指定TTL值。

The TTL index expireAfterSeconds value must be within 0 and 2147483647 inclusive.TTL索引expireAfterSeconds值必须在02147483647之间(包括021474647)。

For example, to create a TTL index on the lastModifiedDate field of the eventlog collection with a TTL value of 3600 seconds, use the following operation in mongosh:例如,要在eventlog集合的lastModifiedDate字段上创建TTL值为3600秒的TTL索引,请在mongosh中使用以下操作:

db.eventlog.createIndex(
{ "lastModifiedDate": 1 },
{ expireAfterSeconds: 3600 }
)

Starting in MongoDB 7.0, you can create partial TTL indexes on time series collections. 从MongoDB 7.0开始,您可以在时间序列集合上创建部分TTL索引。These indexes use the collection timeField as the key field, and require a partial filter expression on the metaField.这些索引使用集合timeField作为键字段,并要求在metaField上使用部分筛选器表达式

Time series collections include an optional expireAfterSeconds field. If you don't set expireAfterSeconds, a TTL index with a partialFilterExpression lets you set an expiration period for documents that match the filter. 时间序列集合包括一个可选的expireAfterSeconds字段。如果不设置expireAfterSeconds,则使用带有partialFilterExpression的TTL索引可以为与筛选器匹配的文档设置过期时间。If you do set expireAfterSeconds, a partial TTL index lets you set a different, shorter expiration period for matching documents. You can only create a partialFilterExpression on the metaField.如果您设置了expireAfterSeconds,部分TTL索引允许您为匹配的文档设置不同的、更短的过期时间。您只能在metaField上创建partialFilterExpression

Important

If the expireAfterSeconds value of the collection is lower than the expireAfterSeconds of the partial TTL index, the collection deletes documents after the shorter time, so the TTL index has no effect.如果集合的expireAfterSeconds值低于部分TTL索引的expireAfterSeconds,则集合会在较短的时间后删除文档,因此TTL索引无效。

This weather data time series collection deletes documents after 24 hours:此天气数据时间序列集合将在24小时后删除文档:

db.createCollection(
"weather24h",
{
timeseries: {
timeField: "timestamp",
metaField: "sensor",
granularity: "hours"
},
expireAfterSeconds: 86400
}
)

This TTL index deletes documents from the MongoDB NYC headquarters weather sensor after 1 hour, instead of 24 hours:此TTL索引在1小时后而不是24小时后从MongoDB NYC总部天气传感器中删除文档:

db.eventlog.createIndex(
{ "timestamp": 1 },
{ partialFilterExpression: { "sensor": { $eq: "40.761873, -73.984287" } } },
{ expireAfterSeconds: 3600 } )

Convert a non-TTL single-field Index into a TTL Index将非TTL单字段索引转换为TTL索引

Starting in MongoDB 5.1, you can add the expireAfterSeconds option to an existing single-field index. To change a non-TTL single-field index to a TTL index, use the collMod database command:从MongoDB 5.1开始,您可以将expireAfterSeconds选项添加到现有的单字段索引中。要将非TTL单字段索引更改为TTL索引,请使用collMod数据库命令:

db.runCommand({
"collMod": <collName>,
"index": {
"keyPattern": <keyPattern>,
"expireAfterSeconds": <number>
}
})

The following example converts a non-TTL single-field index with the pattern { "lastModifiedDate": 1 } into a TTL index:以下示例将模式为{ "lastModifiedDate": 1 }的非TTL单字段索引转换为TTL索引:

db.runCommand({
"collMod": "tickets",
"index": {
"keyPattern": { "lastModifiedDate": 1 },
"expireAfterSeconds": 100
}
})

Change the expireAfterSeconds value for a TTL Index更改TTL索引的expireAfterSeconds

To change the expireAfterSeconds value for a TTL Index, use the collMod database command:要更改TTL索引的expireAfterSeconds值,请使用collMod数据库命令:

db.runCommand({
"collMod": <collName>,
"index": {
"keyPattern": <keyPattern>,
"expireAfterSeconds": <number>
}
})

The following example changes the expireAfterSeconds value for an index with the pattern { "lastModifiedDate": 1 } on the tickets collection:以下示例更改tickets集合上具有模式{ "lastModifiedDate": 1 }的索引的expireAfterSeconds值:

db.runCommand({
"collMod": "tickets",
"index": {
"keyPattern": { "lastModifiedDate": 1 },
"expireAfterSeconds": 100
}
})

Important

Consider the following before updating the expireAfterSeconds parameter of a TTL index:在更新TTL索引的expireAfterSeconds参数之前,请考虑以下事项:

  • Changing the expireAfterSeconds parameter does not trigger a complete index rebuild. However, reducing the expireAfterSeconds value can make many documents eligible for immediate deletion, potentially causing performance issues due to the increased delete operations.更改expireAfterSeconds参数不会触发完整的索引重建。但是,减小expireAfterSeconds值会使许多文档符合立即删除的条件,由于删除操作的增加,可能会导致性能问题。
  • The recommended approach is to manually delete documents in small batches before updating the TTL index. This helps control the impact on your cluster.建议的方法是在更新TTL索引之前手动小批量删除文档。这有助于控制对集群的影响。
  • Deleting many documents can fragment storage files, additionally impacting performance. You may need to run the compact command on your collection or perform a Initial Sync to reclaim space and optimize storage.删除许多文档会使存储文件分片化,还会影响性能。您可能需要在集合上运行compact命令或执行初始同步以回收空间并优化存储。

Behavior行为

Expiration of Data数据到期

TTL indexes expire documents after the specified number of seconds has passed since the indexed field value. The expiration threshold is the indexed field value plus the specified number of seconds.TTL索引在索引字段值后经过指定秒数后使文档过期。过期阈值是索引字段值加上指定的秒数。

If the field is an array, and there are multiple date values in the index, MongoDB uses lowest (earliest) date value in the array to calculate the expiration threshold.如果字段是一个数组,并且索引中有多个日期值,MongoDB将使用数组中最低(最早)的日期值来计算过期阈值。

For time series collections, TTL indexes also remove a bucket of data when all documents inside it expire. 对于时间序列集合,TTL索引还会在其中的所有文档过期时删除一桶数据。This is equal to the upper timestamp limit of the bucket, plus the expireAfterSeconds value. 这等于桶的时间戳上限加上expireAfterSeconds值。For example, if a bucket covers data up until 2023-03-27T18:29:59Z and expireAfterSeconds is 300, the TTL index expires the bucket after 2023-03-27T18:34:59Z.例如,如果一个桶覆盖了2023-03-27T18:29:59Z之前的数据,并且expireAfterSeconds300,则TTL索引将在2023-03-17T18:34:59Z之后过期。

If the indexed field in a document doesn't contain one or more date values, the document will not expire.如果文档中的索引字段不包含一个或多个日期值,则文档不会过期。

If a document does not contain the indexed field, the document will not expire.如果文档不包含索引字段,则文档不会过期。

Delete Operations删除操作

A background thread in mongod reads the values in the index and removes expired documents from the collection.mongod中的后台线程读取索引中的值,并从集合中删除过期的文档

In progress delete operations performed by the TTL thread appear in db.currentOp() output. TTL线程正在执行的删除操作显示在db.currentOp()输出中。As the TTL thread deletes documents, the metrics.ttl.deletedDocuments server status metric is incremented.当TTL线程删除文档时,metrics.ttl.deletedDocuments服务器状态度量会递增。

Starting in MongoDB 6.1:从MongoDB 6.1开始:

  • To improve efficiency, MongoDB may batch multiple document deletions together.为了提高效率,MongoDB可能会同时批量删除多个文档。
  • The explain command results contain a new BATCHED_DELETE stage for batched document deletions.explain命令结果包含一个新的BATCHED_DELETE阶段,用于批量删除文档。

Deletion Process删除过程

The TTL background deletion process checks each TTL index for expired documents. For each TTL index, the background process deletes documents until one of the following conditions is met:TTL后台删除过程检查每个TTL索引是否有过期文档。对于每个TTL索引,后台进程都会删除文档,直到满足以下条件之一:

  • The process deletes 50000 documents from the current index.该过程将从当前索引中删除50000个文档。
  • The process spends one second deleting documents from the current index.该过程花费一秒钟从当前索引中删除文档。
  • All expired documents are deleted from the current index.所有过期的文档都将从当前索引中删除。

Then, the process moves on to the next index. After the process goes through each TTL index once, the current sub-pass is complete and a new sub-pass begins to check for remaining expired documents. A pass is complete when the TTL monitor has deleted all possible candidate documents from all TTL indexes.然后,该过程继续进行到下一个索引。在该过程遍历每个TTL索引一次后,当前子通道完成,新的子通道开始检查剩余的过期文档。当TTL监视器从所有TTL索引中删除了所有可能的候选文档时,传递完成。

Important

The TTL deletion process is a single-threaded background task, meaning that TTL deletions are not concurrent and may take longer under heavy workloads or when processing a large amount of expired documents.TTL删除过程是一个单线程后台任务,这意味着TTL删除不是并发的,在繁重的工作负载或处理大量过期文档时可能需要更长的时间。

Additionally, the process stops the current deletion loop every 60 seconds to prevent spending too much time on a single large delete. When this happens, the current sub-pass ends and a new sub-pass begins.此外,该过程每60秒停止一次当前的删除循环,以防止在单个大型删除上花费太多时间。当这种情况发生时,当前子过程结束,新的子过程开始。

Passes and sub-passes are tracked in the metrics.ttl.passes and metrics.ttl.subPasses server status metrics, respectively.通道和子通道分别在metrics.ttl.passesmetrics.ttl.subPasses服务器状态度量中跟踪。

Timing of the Delete Operation删除操作的时间

MongoDB begins removing expired documents or time series buckets as soon as the index finishes building on the primary. For more information on the index build process, see Index Builds on Populated Collections.一旦索引在primary上构建完成,MongoDB就开始删除过期的文档或时间序列桶。有关索引构建过程的更多信息,请参阅基于填充集合的索引构建。

The TTL index does not guarantee that expired data is deleted immediately upon expiration. There may be a delay between the time that a document expires and the time that MongoDB removes the document from the database.TTL索引不保证过期数据在过期后立即删除。文档过期的时间和MongoDB从数据库中删除文档的时间之间可能存在延迟。

The background task that removes expired documents runs every 60 seconds. As a result, documents may remain in a collection during the period between the expiration of the document and the running of the background task. MongoDB starts deleting documents 0 to 60 seconds after the index completes.删除过期文档的后台任务每60秒运行一次。因此,在文档到期和后台任务运行之间的时间段内,文档可能会保留在集合中。MongoDB在索引完成后0到60秒开始删除文档。

Because the duration of the removal operation depends on the workload of your mongod instance, expired data may exist for some time beyond the 60 second period between runs of the background task.由于删除操作的持续时间取决于mongod实例的工作负载,因此过期数据可能存在一段时间,超过后台任务运行之间的60秒时间。

The delete operations initiated by the TTL task run in the foreground, like other deletes.TTL任务启动的删除操作与其他删除操作一样在前台运行。

Replica Sets复制集

On replica set members, the TTL background thread only deletes documents when a member is in state primary. 副本集成员上,TTL后台线程仅在成员处于主状态时删除文档。The TTL background thread is idle when a member is in state secondary. Secondary members replicate deletion operations from the primary.当成员处于secondary状态时,TTL后台线程处于空闲状态。secondary成员从主要成员复制删除操作。

Support for Queries查询支持

A TTL index supports queries in the same way non-TTL indexes do.TTL索引支持查询的方式与非TTL索引相同。

mongod in Standalone Mode独立模式下的mongod

The TTL monitor stops when mongod runs in standalone mode and the system.local.replset collection contains data. If you take a replica set node out of the replica set and run it as a standalone, then the TTL monitor is disabled.mongod在独立模式下运行并且system.local.replset集合包含数据时,TTL监视器停止。如果从副本集中取出副本集节点并将其作为独立节点运行,则TTL监视器将被禁用。

TTL and Schema validationTTL和模式验证

Even though TTL indexes do not require schema validation, using validation can help ensure consistent behavior by standardizing the existence and format of the date field used for expiration.尽管TTL索引不需要模式验证,但使用验证可以通过标准化用于过期的日期字段的存在和格式来帮助确保一致的行为。

For example, you can use schema validation to enforce the presence of a lastModifiedDate field and ensure its value adheres to a valid date format:例如,您可以使用模式验证来强制lastModifiedDate字段的存在,并确保其值遵循有效的日期格式:

db.createCollection(
"eventlog",
{
validator: {
$jsonSchema: {
bsonType: "object",
required: [ "lastModifiedDate" ],
properties: {
lastModifiedDate: {
bsonType: "date",
description: "Must be a valid date."
}
}
}
}
}
)

This schema validation rule ensures that:此模式验证规则可确保:

  • Every document in the eventlog collection includes the lastModifiedDate field.eventlog集合中的每个文档都包含lastModifiedDate字段。
  • The lastModifiedDate field contains a valid date value.lastModifiedDate字段包含有效的日期值。

Restrictions限制