Migrate Data into a Time Series Collection将数据迁移到时间序列集合中

On this page本页内容

To migrate data from an existing collection into a time series collection:要将数据从现有集合迁移到时间序列集合,请执行以下操作:

  1. Create a New Time Series Collection创建新的时间序列集合
  2. Transform Data (Optional)转换数据(可选)
  3. Migrate Data into a Time Series Collection将数据迁移到时间序列集合中

Create a New Time Series Collection创建新的时间序列集合

To create a new time series collection, issue the following command in the mongosh:要创建新的时间序列集合,请在mongosh中发出以下命令:

db.createCollection(
    "weathernew", {
      timeseries: {
         timeField: "ts",
         metaField: "metaData",
         granularity: "hours"
       }
     }
)

For more information on the preceeding command, see Create a Time Series Collection.有关上述命令的详细信息,请参阅创建时间序列集合

Transform Data (Optional)转换数据(可选)

Time series collections support secondary indexes on the field specified as the metaField. 时间序列集合在指定为metaField的字段上支持二级索引If the data model of your time series data does not have a designated field for your metadata, you can transform your data to create one. 如果时间序列数据的数据模型没有为元数据指定的字段,则可以转换数据以创建一个字段。To transform the data in your existing collection, use $merge or $out to create a temporary collection with your time series data.要转换现有集合中的数据,请使用$merge$out使用时间序列数据创建临时集合。

Consider a collection with weather data of the following format:考虑集合以下格式的天气数据:

 {
    "_id" : ObjectId("5553a998e4b02cf7151190b8"),
    "st" : "x+47600-047900",
    "ts" : ISODate("1984-03-05T13:00:00Z"),
    "position" : {
      "type" : "Point",
      "coordinates" : [ -47.9, 47.6 ]
    },
    "elevation" : 9999,
    "callLetters" : "VCSZ",
    "qualityControlProcess" : "V020",
    "dataSource" : "4",
    "type" : "FM-13",
    "airTemperature" : { "value" : -3.1, "quality" : "1" },
    "dewPoint" : { "value" : 999.9, "quality" : "9" },
    "pressure" : { "value" : 1015.3, "quality" : "1" },
    "wind" : {
      "direction" : { "angle" : 999, "quality" : "9" },
      "type" : "9",
      "speed" : { "rate" : 999.9, "quality" : "9" }
    },
    "visibility" : {
      "distance" : { "value" : 999999, "quality" : "9" },
      "variability" : { "value" : "N", "quality" : "9" }
    },
    "skyCondition" : {
      "ceilingHeight" : { "value" : 99999, "quality" : "9", "determination" : "9" },
      "cavok" : "N"
    },
    "sections" : [ "AG1" ],
    "precipitationEstimatedObservation" : { "discrepancy" : "2", "estimatedWaterDepth" : 999 }
}

To transform this data, we issue the following command:要转换此数据,我们发出以下命令:

db.weather_data.aggregate([
  {
     $addFields: {
       metaData: {
         "st": "$st",
         "position": "$position",
         "elevation": "$elevation",
         "callLetters": "$callLetters",
         "qualityControlProcess": "$qualityControlProcess",
         "type": "$type"
       }
     },
  }, {
     $project: {
        _id: 1,
        ts: 1,
        metaData: 1,
        dataSource: 1,
        airTemperature: 1,
        dewPoint: 1,
        pressure: 1,
        wind: 1,
        visibility: 1,
        skyCondition: 1,
        sections: 1,
        precipitationEstimatedObservation: 1
     }
  }, {
     $out: "temporarytimeseries"
  }
])

After you run this command, you have an intermediary temporarytimeseries collection:运行此命令后,将有一个中间的temporarytimeseries集合:

db.temporarytimeseries.findOne()
{
   "_id" : ObjectId("5553a998e4b02cf7151190b8"),
   "ts" : ISODate("1984-03-05T13:00:00Z"),
   "dataSource" : "4",
   "airTemperature" : { "value" : -3.1, "quality" : "1" },
   "dewPoint" : { "value" : 999.9, "quality" : "9" },
   "pressure" : { "value" : 1015.3, "quality" : "1" },
   "wind" : {
     "direction" : { "angle" : 999, "quality" : "9" },
     "type" : "9",
     "speed" : { "rate" : 999.9, "quality" : "9" }
   },
   "visibility" : {
     "distance" : { "value" : 999999, "quality" : "9" },
     "variability" : { "value" : "N", "quality" : "9" }
   },
   "skyCondition" : {
     "ceilingHeight" : { "value" : 99999, "quality" : "9", "determination" : "9" },
     "cavok" : "N"
   },
   "sections" : [ "AG1" ],
   "precipitationEstimatedObservation" : { "discrepancy" : "2", "estimatedWaterDepth" : 999 },
   "metaData" : {
     "st" : "x+47600-047900",
     "position" : {
       "type" : "Point",
       "coordinates" : [ -47.9, 47.6 ]
     },
     "elevation" : 9999,
     "callLetters" : "VCSZ",
     "qualityControlProcess" : "V020",
     "type" : "FM-13"
   }
}

Migrate Data into a Time Series Collection将数据迁移到时间序列集合中

To migrate your data from an existing collection that is not of type timeseries into a time series collection, use mongodump and mongorestore.要将数据从不属于timeseries类型的现有集合迁移到时间序列集合,请使用mongodumpmongorestore

Warning警告

When migrating or backfilling into a time series collection you should always insert the documents in order, from oldest to newest. 当迁移或回填到时间序列集合中时,您应该始终按顺序插入文档,从最旧到最新。In this case mongodump exports documents in natural order and the --maintainInsertionOrder option for mongorestore guarantees the same insertion order for documents.在这种情况下,mongodump以自然顺序导出文档,mongorestore--maintainInsertionOrder选项保证文档的插入顺序相同。

For example, to export the temporarytimeseries collection, issue the following command:例如,要导出temporarytimeseries集合,请发出以下命令:

mongodump
    --uri="mongodb://mongodb0.example.com:27017,mongodb1.example.com:27017,mongodb2.example.com:27017/weather" \
    --collection=temporarytimeseries --out=timeseries

The command returns the following output:该命令返回以下输出:

2021-06-01T16:48:39.980+0200  writing weather.temporarytimeseries to timeseries/weather/temporarytimeseries.bson
2021-06-01T16:48:40.056+0200  done dumping weather.temporarytimeseries (10000 documents)

To import timeseries/weather/temporarytimeseries.bson into the new collection weathernew, issue the following command:要将timeseries/weather/temporarytimeseries.bson导入新集合weathernew,请发出以下命令:

mongorestore
    --uri="mongodb://mongodb0.example.com:27017,mongodb1.example.com:27017,mongodb2.example.com:27017/weather" \
    --collection=weathernew --noIndexRestore \
    --maintainInsertionOrder \
    timeseries/weather/temporarytimeseries.bson

The command returns the following output:该命令返回以下输出:

2021-06-01T16:50:56.639+0200  checking for collection data in timeseries/weather/temporarytimeseries.bson
2021-06-01T16:50:56.640+0200  restoring to existing collection weather.weathernew without dropping
2021-06-01T16:50:56.640+0200  reading metadata for weather.weathernew from timeseries/weather/temporarytimeseries.metadata.json
2021-06-01T16:50:56.640+0200  restoring weather.weathernew from timeseries/weather/temporarytimeseries.bson
2021-06-01T16:51:01.229+0200  no indexes to restore
2021-06-01T16:51:01.229+0200  finished restoring weather.weathernew (10000 documents, 0 failures)
2021-06-01T16:51:01.229+0200  10000 document(s) restored successfully. 0 document(s) failed to restore.
Note注意

Ensure that you run the preceeding command with the --noIndexRestore option. 确保使用--noIndexRestore选项运行前面的命令。mongorestore cannot create indexes on time series collections.无法在时间序列集合上创建索引。

If your original collection had secondary indexes, manually recreate them now.如果原始集合有二级索引,请立即手动重新创建它们。

←  Add Secondary Indexes on metaField and timeFieldBuild Materialized Views on Top of Time Series Data →