| • |
|
|
|
|||
|
||||
The meeting was called by NCDC so they could gather information on how the NOAAPort data stream will affect their data archiving. There were two main goals. First, learn more about the NOAAPort data stream. This included what datasets are in the data stream, the volume of the data, and the formats of the data. Second, and most important, start determining the requirements of the users. Mainly, this includes what data is required by the users and how long that data needs to be archived. The two main user groups are NWS users and lawyers (getting data for litigations).
We were given a tour of NCDC following the path data follows. It started with ingesting the data. Some data are still received in analog formats. This requires hand entering and digitizing the data. Most of the data volume is received in digital form, on tape or downloaded. One dataset is retrieved through modem calls to 400-500 sensor sites. A big benefit for NCDC from NOAAPort will be the reduced number of data source sites. With NOAAPort, one input source will supply many of it's data needs.
After ingest, the data goes through an extensive quality control process. Software to do the QC checks has been and is being written. However, there are still pieces of the QC that have to be done by a person.
In the tape and computer rooms, NCDC is continually modernizing. They move their archive to a new storage media every 5-7 years (this time period is shrinking as technology advances speed up).
Of course, all the users want all the data for all times in whatever format they want to use. However, because archiving costs money a balance needs to be found between wants, needs, and cost. Most of the discussion about requirements was aimed at listing different levels of data requirements from worst case needs on up.
Andy Edman gave a suggestion for an archiving strategy. It consisted of two separate archives. First, a short term (30 days was discussed) archive that contains all of the data from the NOAAPort data stream. This archive would not be quality controlled other than figuring out what data was there. The format of the data in this archive could be either direct from the NOAAPort data stream or the decoded WFO-Advanced netCDF format. Second, a long term archive that would not necessarily contain all the NOAAPort data. This archive would go through quality control. The format should be as raw as possible, e.g., NCEP models in their native grids rather than interpolated to the AWIPS grids.
The native grid model data are not planned to flow through NOAAPort; this means that NCDC would have to get the data directly from NCEP. NCDC expressed interest in limiting the number of access points from which they would have to get data. However, if that was the only way to meet user requirements they would find a way.
A lot of discussion involved what formats the data should be archived in. One of the action items, listed below, focuses on formats for the short term archive.
One of NCDC's largest customer groups is the legal community. Lawyers and consulting meteorologists require official weather information for use in litigations. The SRRS (?) dataset archive, kept for five years, was developed to satisfy these requirements. There was some discussion about what the SRRS requirements actually are and how to meet those requirements with NOAAPort data.
| Contact Us • Site Map • Search • Terms and Conditions • Privacy Policy • Participation Policy | |||||
|
|||||
