| • |
|
|
|
|||
|
||||
This document is intended to be a start of information regarding data useful to LEAD. I have tried to make it as accurate as possible, but much of this information is most accessible via word of mouth and things change regularly. I expect this to be a work in progress to be corrected and expanded as new information becomes available and as LEAD moves in to include additional types of data. I have also included relevant information that I found useful in understanding the data from my perspective as a non-meteorologist.
The Unidata LDM feed types page contains information and useful links about the data relayed via our IDD. Also, the Unidata Real-Time IDD Statistics page shows real time product count and volume statistics by host or by feed type. In particular, our main server here in Boulder, thelma.ucar.edu, recieves nearly all the data discussed in this document (except ACARS, which is small), so that's a good one to view for LEAD purposes.
Format: Text. Raw format encodings are described in the Surface Weather Observations and Reports FMH (Federal Meteorological Handbook) No. 1 chapter 12. Additional encoding information is available in this page on WMO International Codes .
Variables: Contains textual encodings to describe:
Product size: very small, often < 1K. Occasionally up to 5K. Note that these are bulletins that may contain multiple METARs.
Volume: A sample collection period of roughly one day yielded about 113MB.
Unidata Feed Type: IDS|DDPLUS, SA* and SP* (for surface)
Other: METAR format is one format used to transmit surface observations. Synoptic code is another format for surface observations. Synoptic code bulletins include land surface, ship, buoy, and CMAN (Coastal Marine Automated Network) stations. It is expected that ship and buoy information will be required by LEAD. These are available via the IDS|DDPLUS feed matching SI*, SX*, SM*, and SN*. This Surface Weather Reports from land and sea stations report gives useful information regarding these bulletins.
Format: Text
Variables: Contains textual encodings to describe:
Independent of the rate at which balloons generate data, bulletins are sent at X hour intervals (?), with the large majority of reports appearing at 0Z and 12Z.
Product size: very small, vast majority <1K, occasionally up to 25K. Note that these are bulletins that may contain multiple upper air reports.
Volume: About 6MB/day.
Unidata Feed Type: IDS|DDPLUS, U* (for upper air)
Other: NWS Rawinsonde Network map. Also, here is an information page for students. More info available here and here.
Format: gzipped netCDF
Variables - point samples (not averages):
At FSL data may be gathered into a one hour moving window. (?)
Product size: 100K to 700K.
Volume: about 1.8 megabytes per hour in netCDF format
Unidata Feed Type: ACARS (aka PCWS)
For general WSR-88D information, see this page. Both the Level II and Level III radar products come from these radars.
Format: (Paraphrased and updated by Anne from text written by Harry Edmon): The data format is the same as the level II tapes data files from NCDC. Each volume scan is broken up into smaller products and compressed. Each product consists of a 4 byte record size in bytes (big-endian), followed by the data record itself. The data record consists of data compressed using bzip2. The first product contains the 24 byte volume header and the first 100 radials. All subsequent products contain up to 100 radials of data. Products will be ended before 100 radials when there is an end of elevation.
Generation Rate: depends on mode of the radar. A complete volume takes 4.7 minutes to generate. Two storm modes generate volumes every 5 or 6 minutes respectively, and clear air mode generates a volume every 10 minutes.
Variables:
Volume:
Compressed: A 26 hour sample of volumes from 92 radars yielded about
5.9 gigabytes. This gives 2.4Mb per hour per radar.
Uncompressed: A 24 hour sample give about 135 uncompressed gigabytes
for 99 radars. This yields 56MB per hour per radar.
These combined results give an average 23.33/1 compression ratio.
Note that these are current volumes, but as the NWS builds up the CRAFT network new stations are coming on line. Ultimately there will be 132 radars participating, and the number may increase to 140. Additionally, radars are expected to include more tilts, which also contribute to an increase in volume. NWS estimates a total maximum possible volume in 2005 of 432GB/day.
This volume will vary with the weather.
Unidata Feed Type: CRAFT (aka NEXRD2)
Other: We sometimes refer to this data as CRAFT (Collaborative Radar Acquisition Field Test) data, an acronym created when Kelvin's group was first testing the concept of relaying this data. Additional information is available at the CRAFT page.
The lowest four tilts initially required by LEAD are the products N0R, N1R, N2R, and N3R, which show base reflectivity.
Format: zlib compressed binary. Content can be either polar coordinates or rasterized for display, depending on product.
Variables: NxR products depict base reflectivity. Please see Level III web pages above for more information.
Product size: 2K to 20K.
Generation Rate: Every five, six, or ten minutes, depending on the mode of the radar.
Volume: A 26 hour sample of N0R products from all radars yielded about 422 megabytes. These products are expected to be the largest of the four, as all precipitation falls downward. These volumes will vary with the weather.
Unidata Feed Type: NEXRAD
GOES imagery is available from two sources:
Variables: One band (wave length channel) per AREA file. Bands available include
Generation Rate: Currently three channels are available via IDD (vis, water vapor, and thermal IR). Each generates an image every hour. IDD relay will include all five channels twice per hour by the end of January.
Product Size: see Unidata-Wisconsin Data stream Changes (these are no longer proposed but to be implemented by the end of January).
Volume: Compressed volume is currently 6MB/hour (for three channels). The increased number of channels and frequency will give 25MB/hour.
Unidata feed type: UNIWISC.
Variables: 1 band per image, same bands as above.
Generation Rate: Conus images generated every 15 minutes.
Product Size: Compressed: < 18MB. Uncompressed: visible is a26MB, all others are smaller.
Volume:
Compressed: typically ranges from 50 to 225MB. (This
varies diurnally.)
Uncompressed: 26 * 4 * 2 (for both East and West) = 208MB/hour.
Unidata Feed Type: NIMAGE
Format: GRIB. Grid points are mapped to a projection. (Both 212 and 218 use Lambert Conformal.) Each grid/bulletin represents a field/level/time combination. Each grid point represents a location in grid space and contains a field value.
Variables: The ETA model produces hundreds of fields. To see the fields go to Operational Eta and click on "File Inventory" for the desired grid number. The ETA 40 km 212 is the highest resolution data set available on CONDUIT with the fields necessary for mesoscale modeling at this time. Unidata expects to have the 218 (12 km) grid available in a month or two.
Generation Rate: Model is run every six hours, creates forecasts in three-hourly steps, out to 84 hours.
Product size: Unidata breaks up the model output into separate grids. Grid pieces range in size from about 94 bytes to 200K bytes.
Volume: The 212 generates 14.3MB per forecast hour, with 29 forecast hours (84 hours out at three hourly timesteps), giving roughly 415MB/run, with four runs per day.
Unidata Feed Type: CONDUIT (full model output), HDS (fewer fields)
Other: NCEP Eta Information, ETA FAQ
| Contact Us • Site Map • Search • Terms and Conditions • Privacy Policy • Participation Policy | ||||||
|
||||||
