[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20030422: GEM regional model data from CMC on LDM



>From: "Toth,Garry [CMC]" <address@hidden>
>Organization: CMC
>Keywords: 200304222039.h3MKdh7U007670 IDD GEM

Garry,

>     I think Lewis Poulin from CMC (Canadian Meteorological Centre) has
>contacted you recently about some technical issues.  I 
>am a CMC meteorologist on part-time assignment at COMET.
>
>     I am writing to inform you that CMC is planning to increase the volume
>of data we send to the LDM.  The main increase will be 
>due to many more vertical levels of data.  We estimate the data volume could
>increase by a factor of 3 to 4.  We would do this 
>some time in May.  Would this pose any problem for you at Unidata?

This does not pose any problem for Unidata, the LDM, or the IDD.
It would be a good idea, however, to prepare an announcement of the
change so that it can be sent out to the users of the GEM model data.
I would include what you said above, but with a little more technical
detail.  Also, you should probably include an estimate for the increase
in data.  You can do this by taking your increase factor and multiplying
it by the 'volume' numbers you can gleen from the real time statistics
information found for the GEM stream in:

http://www.unidata.ucar.edu/staff/chiz/rtstats/siteindex.shtml?thelma.ucar.edu

>     I also have a question somewhat more technical in nature.  You obtain
>our CMC data, I think, from the following ftp site:
>
>address@hidden:/data/dns2/ldm  .

As far as I know, we only get the data by LDM transfer, not by FTP.

>     The people currently responsible here at CMC for getting our data to
>this site noticed that we send a large number of individual files: 
>there is one file for each model variable, for each forecast time, and for
>each model level.  This makes 475 files for each 12 hour period.

Each GRIB message is sent out as an individual IDD product.  This was
done by design.

>     Those who originally set this up (around  Feb, 2002, I think) are no
>longer here, so the question was posed to me:  Why do we send 
>such a large number of separate files?  I don't know the answer, so I am
>hoping you can help me with it.  Do you know why the system 
>was set up that way?  Is it necessary to do that, or could the data files be
>combined in some way?  For example, would it be possible 
>to send all the 6-hr forecasts in a single P06 file?

While the LDM can easily send large data files, it is preferable to
send the individual GRIB messages as individual products.  This allows
the downstream sites (i.e., the data users) to be able to request just
the data they want (e.g., just the temperature fields, or just the wind
fields, etc.) and perform actions based on the receipt of those
individual fields.  This is the same way that we send the high
resolution model output from NCEP in a stream we call CONDUIT, and we
would like to keep the GEM data the same.

>      Thanks for your consideration of these questions.

Please let us know if the above explanation was not detailed enough.

Cheers,

Tom Yoksas