[ldm-users] CONDUIT slow?

[Gerry Creager]

Chiz,

Thanks for the analysis.  The basis for my comment is that I'd 
historically been seeing the bulk of the GFS 0.5 deg. grib2's on our 
system by about 3:13 past the nominal forecast start time for about a 4 
week period. I used this as a baseline, and scripted up a process that 
starts via cron at 13 min past the appropriate hour and checks for 
existence of the final file needed, and sleeps one minute, tests and 
loops if not present, until the file's there.  It then kicks off WRF 
preprocessing code.

Lately, I've seen the time start creeping up toward 3:45-4:01 post start 
time.  It doesn't cause me particular grief but does cause me to notice 
the event.  Just thought I'd ask.

Regards, Gerry

Steve Chiswell wrote:
> Gerry,
> 
> It appears to me that the files being inserted on CONDUIT are timely
> with
> the data being posted on the ftp server where the data originates.
>  Eg, there is no delay in posting of the data to the LDM system feeding
> CONDUIT. 
> Also, your rtstats appear to show your system receiving the data
> without latency.
> 
> As an example, the 12Z F180 0.5 degree and 1.0 degree .status files
> for the past 2 days show consistent posting into CONDUIT at 16:34Z
> 
> Sep 25 12Z
> gfs.2007092512/gfs.t12z.pgrb2f180:data/nccf/com/gfs/prod/gfs.2007092512/gfs.t12z.pgrb2f180 complete (51162456 bytes) at Tue Sep 25 16:34:04 2007
> gfs.2007092512/gfs.t12z.pgrbf180:data/nccf/com/gfs/prod/gfs.2007092512/gfs.t12z.pgrbf180 complete (30171092 bytes) at Tue Sep 25 16:33:44 2007
> 
> Sep 26 12Z
> gfs.2007092612/gfs.t12z.pgrb2f180:data/nccf/com/gfs/prod/gfs.2007092612/gfs.t12z.pgrb2f180 complete (51533295 bytes) at Wed Sep 26 16:34:16 2007
> gfs.2007092612/gfs.t12z.pgrbf180:data/nccf/com/gfs/prod/gfs.2007092612/gfs.t12z.pgrbf180 complete (30117174 bytes) at Wed Sep 26 16:34:07 2007
> 
> The ftpprd ftp server shows
> for /pub/data/nccf/com/gfs/prod/gfs.2007092612 a similar 16:34Z posting
> time:
> 
> ftp> dir gfs.t12z.pgrbf180
> -rwxrwxr-x   1 ftp      ftp      30117174 Sep 26 16:34 gfs.t12z.pgrbf180
> 
> ftp>  dir gfs.t12z.pgrb2f180
> -rw-r--r--   1 ftp      ftp      51533295 Sep 26 16:33 gfs.t12z.pgrb2f180
> 
> 
> If you have specific times you would like to check in to, for data
> availability, please let me know. If you are not seeing your data
> locally around the times noted above, then perhaps you need to look at
> your pqact latency on your LDM to ensure that 
> your system isn't falling behind in its ability to keep up with the IO.
> 
> Steve Chiswell
> Unidata User Support
> 
> 
> On Wed, 2007-09-26 at 13:39 -0500, Gerry Creager wrote:
>> Files are all consistent in size.  I don't recall the exact pqact entry 
>> but I think they do overwrite.
>>
>> Who'd be requesting retransmissions from this group?  I went to CONDUIT 
>> to avoid having to hit the TOC and WOC gateways...
>>
>> Gilbert Sebenste wrote:
>>> On Wed, 26 Sep 2007, Gerry Creager wrote:
>>>
>>>> Has anyone noted GFS 0.5deg is coming in later and later via CONDUIT?
>>> According to the NWS Gateway message sent out late yesterday, someone has 
>>> been requesting retransmissions of the data. Check to see if your files 
>>> are larger than normal, unless they overwrite automagically...
>>>
>>> *******************************************************************************
>>> Gilbert Sebenste                                                     ********
>>> (My opinions only!)                                                  ******
>>> Staff Meteorologist, Northern Illinois University                      ****
>>> E-mail: sebenste at weather.admin.niu.edu                                  ***
>>> web: http://weather.admin.niu.edu                                      **
>>> *******************************************************************************
>>> _______________________________________________
>>> ldm-users mailing list
>>> ldm-users at unidata.ucar.edu
>>> For list information or to unsubscribe,  visit: http://www.unidata.ucar.edu/mailing_lists/ 

-- 
Gerry Creager -- gerry.creager at tamu.edu
Texas Mesonet -- AATLT, Texas A&M University
Cell: 979.229.5301 Office: 979.862.3982 FAX: 979.862.3983
Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843