Re: [ldm-users] CONDUIT slow?


Thanks for the analysis. The basis for my comment is that I'd historically been seeing the bulk of the GFS 0.5 deg. grib2's on our system by about 3:13 past the nominal forecast start time for about a 4 week period. I used this as a baseline, and scripted up a process that starts via cron at 13 min past the appropriate hour and checks for existence of the final file needed, and sleeps one minute, tests and loops if not present, until the file's there. It then kicks off WRF preprocessing code.

Lately, I've seen the time start creeping up toward 3:45-4:01 post start time. It doesn't cause me particular grief but does cause me to notice the event. Just thought I'd ask.

Regards, Gerry

Steve Chiswell wrote:

It appears to me that the files being inserted on CONDUIT are timely
the data being posted on the ftp server where the data originates.
 Eg, there is no delay in posting of the data to the LDM system feeding
CONDUIT. Also, your rtstats appear to show your system receiving the data
without latency.

As an example, the 12Z F180 0.5 degree and 1.0 degree .status files
for the past 2 days show consistent posting into CONDUIT at 16:34Z

Sep 25 12Z
 complete (51162456 bytes) at Tue Sep 25 16:34:04 2007
 complete (30171092 bytes) at Tue Sep 25 16:33:44 2007

Sep 26 12Z
 complete (51533295 bytes) at Wed Sep 26 16:34:16 2007
 complete (30117174 bytes) at Wed Sep 26 16:34:07 2007

The ftpprd ftp server shows
for /pub/data/nccf/com/gfs/prod/gfs.2007092612 a similar 16:34Z posting

ftp> dir gfs.t12z.pgrbf180
-rwxrwxr-x   1 ftp      ftp      30117174 Sep 26 16:34 gfs.t12z.pgrbf180

ftp>  dir gfs.t12z.pgrb2f180
-rw-r--r--   1 ftp      ftp      51533295 Sep 26 16:33 gfs.t12z.pgrb2f180

If you have specific times you would like to check in to, for data
availability, please let me know. If you are not seeing your data
locally around the times noted above, then perhaps you need to look at
your pqact latency on your LDM to ensure that your system isn't falling behind in its ability to keep up with the IO.

Steve Chiswell
Unidata User Support

On Wed, 2007-09-26 at 13:39 -0500, Gerry Creager wrote:
Files are all consistent in size. I don't recall the exact pqact entry but I think they do overwrite.

Who'd be requesting retransmissions from this group? I went to CONDUIT to avoid having to hit the TOC and WOC gateways...

Gilbert Sebenste wrote:
On Wed, 26 Sep 2007, Gerry Creager wrote:

Has anyone noted GFS 0.5deg is coming in later and later via CONDUIT?
According to the NWS Gateway message sent out late yesterday, someone has been requesting retransmissions of the data. Check to see if your files are larger than normal, unless they overwrite automagically...

Gilbert Sebenste                                                     ********
(My opinions only!)                                                  ******
Staff Meteorologist, Northern Illinois University                      ****
E-mail: sebenste@xxxxxxxxxxxxxxxxxxxxx                                  ***
web:                                      **
ldm-users mailing list
For list information or to unsubscribe, visit:

Gerry Creager -- gerry.creager@xxxxxxxx
Texas Mesonet -- AATLT, Texas A&M University
Cell: 979.229.5301 Office: 979.862.3982 FAX: 979.862.3983
Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843

  • 2007 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: