Database Manual / Time Series / Create & Configure / Migrate Data

Migrate Data into a Time Series Collection with Database Tools使用数据库工具将数据迁移到时间序列集合中

Use the following steps to migrate data from an existing collection to a time series collection with mongodump and mongorestore.使用以下步骤使用mongodump和mongorestore将数据从现有集合迁移到时间序列集合。

Steps步骤

1

Create a new time series collection.创建新的时间序列集合。

To create a new time series collection, issue the following command in the mongosh:要创建新的时间序列集合，请在mongosh中发出以下命令：

db.createCollection(
   "weathernew", {
      timeseries: {
         timeField: "ts",
         metaField: "metaData",
         granularity: "hours"
      }
   }
)

This example uses sample data for the timeField, metaField, and granularity. 此示例使用timeField、metaField和granularity的示例数据。For more information on the preceeding command, see Create a Time Series Collection.有关上述命令的详细信息，请参阅创建时间序列集合。

2

(Optional) Transform your data.（可选）转换数据。

Time series collections support secondary indexes on the field specified as the metaField. 时间序列集合支持对指定为metaField的字段进行辅助索引。If the data model of your time series data does not have a designated field for your metadata, you can transform your data to create one. To transform the data in your existing collection, use $out to create a temporary collection with your time series data.如果时间序列数据的数据模型没有为元数据指定字段，则可以转换数据以创建一个字段。要转换现有集合中的数据，请使用$out创建一个包含时间序列数据的临时集合。

Consider a collection with weather data of the following format:考虑以下格式的天气数据集合：

db.weatherdata.insertOne(
   {
      _id: ObjectId("5553a998e4b02cf7151190b8"),
      st: "x+47600-047900",
      ts: ISODate("1984-03-05T13:00:00Z"),
      position: {
         type: "Point",
         coordinates: [ -47.9, 47.6 ]
      },
      elevation: 9999,
      callLetters: "VCSZ",
      qualityControlProcess: "V020",
      dataSource: "4",
      type: "FM-13",
      airTemperature: { value: -3.1, quality: "1" },
      dewPoint: { value: 999.9, quality : "9" },
      pressure: { value: 1015.3, quality: "1" },
      wind: {
         direction: { angle: 999, quality: "9" },
         type: "9",
         speed: { rate: 999.9, quality: "9" }
      },
      visibility: {
         distance: { value: 999999, quality : "9" },
         variability: { value: "N", quality: "9" }
      },
      skyCondition: {
         ceilingHeight: { value: 99999, quality: "9", determination: "9" },
         cavok: "N"
      },
      sections: [ "AG1" ],
      precipitationEstimatedObservation: {
         discrepancy: "2",
         estimatedWaterDepth: 999
      }
   }
)

Note

Choosing the right field as your time series metaField and grandularity optimizes both storage and query performance. 选择正确的字段作为时间序列metaField和grandularity可以优化存储和查询性能。For more information on field selection and best practices, see metaField and Granularity Best Practices.有关字段选择和最佳实践的更多信息，请参阅metaField和Granularity最佳实践。

The pipline below performs the following operations:下面的管线执行以下操作：

  • Uses $addFields to add a metaData field to the weather_data collection.使用$addFields将元数据字段添加到weather_data集合中。
  • Uses $project to include or exclude the remaining fields in the document.使用$project包含或排除文档中的其余字段。
  • Uses $out to create a temporary collection called temporarytimeseries.使用$out创建一个名为temporarytimeseries的临时集合。
db.weather_data.aggregate([
{
   $addFields: {
      metaData: {
         "st": "$st",
         "position": "$position",
         "elevation": "$elevation",
         "callLetters": "$callLetters",
         "qualityControlProcess": "$qualityControlProcess",
         "type": "$type"
      }
   },
}, {
   $project: {
      _id: 1,
      ts: 1,
      metaData: 1,
      dataSource: 1,
      airTemperature: 1,
      dewPoint: 1,
      pressure: 1,
      wind: 1,
      visibility: 1,
      skyCondition: 1,
      sections: 1,
      precipitationEstimatedObservation: 1
   }
}, {
   $out: "temporarytimeseries"
}
])

After you run this command, you have an intermediary temporarytimeseries collection:运行此命令后，您将获得一个中间的temporarytimeseries集合：

db.temporarytimeseries.findOne()
{
   "_id" : ObjectId("5553a998e4b02cf7151190b8"),
   "ts" : ISODate("1984-03-05T13:00:00Z"),
   "dataSource" : "4",
   "airTemperature" : { "value" : -3.1, "quality" : "1" },
   "dewPoint" : { "value" : 999.9, "quality" : "9" },
   "pressure" : { "value" : 1015.3, "quality" : "1" },
   "wind" : {
   "direction" : { "angle" : 999, "quality" : "9" },
   "type" : "9",
   "speed" : { "rate" : 999.9, "quality" : "9" }
   },
   "visibility" : {
   "distance" : { "value" : 999999, "quality" : "9" },
   "variability" : { "value" : "N", "quality" : "9" }
   },
   "skyCondition" : {
   "ceilingHeight" : { "value" : 99999, "quality" : "9", "determination" : "9" },
   "cavok" : "N"
   },
   "sections" : [ "AG1" ],
   "precipitationEstimatedObservation" : { "discrepancy" : "2", "estimatedWaterDepth" : 999 },
   "metaData" : {
   "st" : "x+47600-047900",
   "position" : {
      "type" : "Point",
      "coordinates" : [ -47.9, 47.6 ]
   },
   "elevation" : 9999,
   "callLetters" : "VCSZ",
   "qualityControlProcess" : "V020",
   "type" : "FM-13"
   }
}
3

Export your original collection.导出原始集合。

To export your data from an existing collection that is not of type timeseries use mongodump.要从非timeseries类型的现有集合导出数据，请使用mongodump。

Warning

When migrating or backfilling into a time series collection, always insert the documents in order, from oldest to newest. 在迁移或回填到时间序列集合中时，请始终按从旧到新的顺序插入文档。In this case, mongodump exports documents in natural order and the --maintainInsertionOrder option for mongorestore guarantees the same insertion order for documents.在这种情况下，mongodump以自然顺序导出文档，mongorestore的--maintainInsertionOrder选项保证文档的插入顺序相同。

For example, to export the temporarytimeseries collection, issue the following command:例如，要导出temporarytimeseries集合，请发出以下命令：

mongodump
   --uri="mongodb://mongodb0.example.com:27017,mongodb1.example.com:27017,mongodb2.example.com:27017/weather" \
   --collection=temporarytimeseries --out=timeseries

The command returns the following output:该命令返回以下输出：

2021-06-01T16:48:39.980+0200  writing weather.temporarytimeseries to timeseries/weather/temporarytimeseries.bson
2021-06-01T16:48:40.056+0200  done dumping weather.temporarytimeseries
(10000 documents)
4

Import your collection.导入集合。

To import your data into a timeseries collection, use mongorestore.要将数据导入时间序列集合，请使用mongorestore。

Important

Ensure that you run the mongorestore command with the --noIndexRestore option. mongorestore cannot create indexes on time series collections.确保使用--noIndexRestore选项运行mongorestore命令。mongorestore无法对时间序列集合创建索引。

The following operation imports timeseries/weather/temporarytimeseries.bson into the new collection weathernew:以下操作将timeseries/weather/temporarytimeseries.bson导入到新集合weathernew中：

mongorestore
   --uri="mongodb://mongodb0.example.com:27017,mongodb1.example.com:27017,mongodb2.example.com:27017/weather" \
   --collection=weathernew --noIndexRestore \
   --maintainInsertionOrder \
   timeseries/weather/temporarytimeseries.bson

The command returns the following output:该命令返回以下输出：

2021-06-01T16:50:56.639+0200  checking for collection data in timeseries/weather/temporarytimeseries.bson
2021-06-01T16:50:56.640+0200  restoring to existing collection weather.weathernew without dropping
2021-06-01T16:50:56.640+0200  reading metadata for weather.weathernew from timeseries/weather/temporarytimeseries.metadata.json
2021-06-01T16:50:56.640+0200  restoring weather.weathernew from timeseries/weather/temporarytimeseries.bson
2021-06-01T16:51:01.229+0200  no indexes to restore
2021-06-01T16:51:01.229+0200  finished restoring weather.weathernew (10000 documents, 0 failures)
2021-06-01T16:51:01.229+0200  10000 document(s) restored successfully. 0 document(s) failed to restore.

If your original collection had secondary indexes, manually recreate them now. 如果原始集合有辅助索引，请立即手动重新创建它们。If your collection includes timeField values before 1970-01-01T00:00:00.000Z or after 2038-01-19T03:14:07.000Z, MongoDB logs a warning and disables some query optimizations that make use of the internal clustered index. 如果集合包含1970-01-01T00:00:00.000Z之前或2038-01-1T03:14:07.000Z之后的timeField值，MongoDB会记录警告并禁用一些使用内部聚集索引的查询优化。Create a secondary index on the timeField to regain query performance and resolve the log warning.在timeField上创建辅助索引以恢复查询性能并解决日志警告。

Tip

Add Secondary Indexes to Time Series Collections向时间序列集合添加二级索引

If you insert a document into a collection with a timeField value before 1970-01-01T00:00:00.000Z or after 2038-01-19T03:14:07.000Z, MongoDB logs a warning and prevents some query optimizations from using the internal index. Create a secondary index on the timeField to regain query performance and resolve the log warning.如果将文档插入到1970-01-01T00:00:00.000Z之前或2038-01-1T03:14:07.000Z之后具有timeField值的集合中，MongoDB会记录警告并阻止某些查询优化使用内部索引。在timeField上创建辅助索引以恢复查询性能并解决日志警告。