[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20050913: Re: [Fwd: Storing only completed gribs?]

Hi Geoff,
--On Tuesday, September 13, 2005 7:42 AM -0400 Geoff DiMego <address@hidden> wrote: 


Linda:

Longtime no see - hope all is well.  I have a question.


That's for sure!  Maybe you'll be at AMS this year, eh?



I thought CONDUIT was only available to universities?
That's not quite true. For example, data are available through quid-pro-quo arrangements from universities who are receiving the data and willing to assist the site and support the request. We are quite sure that this is probably the case with the 3tiergroup.com referred to in the email below. We do not know for sure, because there are no statics provided back to Unidata in such an arrangement. Unidata does not support private industry directly, but indirectly there are email lists, and soon we'll have a Web support system that will provide another level of support for data streams, etc. 

3Tier

sounds like a private sectore entity.  I'm NOT complaining --
just checking.  I get lots of questions about access to our data
from the private sector and, if the private sector can use
CONDUIT, then I can start pointing folks at it/you when they want
an alternative access route to our public ones (which, as you
know, can get strained if there is extra heavy accessing going on).
We are aware of NCEP's issue of bogging down with heavy use, as pointed out by Louis on several occasions. You can have people contact Unidata support <address@hidden> if they have questions about finding a university site who might be willing to work out an arrangement with them. 

Thanks, Geoff, and don't be a stranger.

Linda
P.S. I've copied Mohan, Steve Chiswell and support so they can comment and/or be aware of the issue in the event we have inquiries. 




Thanks in advance.

-------- Original Message --------
Subject: Storing only completed gribs?
Date: Mon, 12 Sep 2005 22:09:30 +0000
From: <address@hidden>
Organization: UCAR/Unidata
To: address@hidden


Hello list,

We're currently pulling in GFS grids over the CONDUIT feed.  They are
stored piece by piece into flat files by pqact using the FILE action

CONDUIT
^/afs/\.nwstg\.nws\.noaa\.gov/ftp/SL\.us008001/ST\.opnl/MT\.(gfs|avn)_CY.([0-9][0-
9])/RD\.([0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9])/PT\.grid_DF\.gr1/fh\.0([0-9][0-
9][0-9])_tl\.press_gr\.onedeg.*$
FILE    data/avn/\3\2/avngrib.\3\2.f0\4

Processes which use this data poll the directories containing those flat
files, looking for new data.  They fail, however, if that file doesn't
have a complete set of data yet.   Is there a common strategy for indicating
that a grib has come in completely?   My only thought right now is to
EXEC to a script which runs GRIB utilities to find out if a
complete set has come in, and only when complete store the file in the final
destination.

Any help would be appreciated.  Thanks!

Joseph Sheedy
Technical Specialist
3TIER Environmental Forecast Group
address@hidden
(206)325-1573 x116






Linda Miller - address@hidden
Community Services, Unidata
University Corporation for Atmospheric Research
P.O. Box 3000
Boulder, CO 80307-3000
303-497-8646 fax: 303-497-8690