[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

19990825: Missing CONDUIT data

Jim,

I think the problem relates to the latency of the data upstream.

The latencies for products arriving at unidata on the NMC2 feed
is available at:
http://www.unidata.ucar.edu/staff/chiz/ncep_idd/gifs/latency.gif

In general, the latency approaches 1 hour 2x per day, one for
the 00Z MRF & AVN runs and the second for the 12Z AVN.

The size of the AVN (600MB) pushes the T-1 capacity (540 Mb/hour)
which pushes the latencies up and is the biggest culprite.
The log messages associated with this should be RECLASS messages if
the LDM is having to jump ahead.

At present, the system uses the same 3600 second latency request
as the other IDD feeds....a tricky situation for LDM's that would receive 
more than NMC2 feed.

This will be an issue to address- either by more bandwith at the top, 
longer queue lives, or decreased data volume injection.

Steve Chiswell 





>From: Jim Cowie <address@hidden>
>Organization: .
>Keywords: 199908251645.KAA25155

>
>This probably goes to Chiz...
>
>I'm trying to track down some missing CONDUIT (NMC2) model data.
>It appears that there are many times that we do not receive the
>entire set of grids. I'm pretty sure it's not an LDM (pq) problem
>on our end and was wondering if someone can trace it back upstream
>to find out what's going on. 
>
>The current topology as I understand it is:
>
>NCEP --> Goddard --> Unidata --> RAP (iita2,australis) --> us
>
>We receive and file *only* the MRF grid 002 and 003 data using a
>600MB pq. There are no messages in our ldm log file complaining
>about the queue being filled up, so I feel pretty good about our end. 
>When data is missing here, it is also missing at RAP so the problem
>must be upstream from RAP.
>
>Here is a list of the raw grib data file sizes (bytes) for the past
>5 days for grid 002 and 003:
>
>Date           Grid 2 size     Grid 3 size
>-------------------------------------------
>19990821       119740886       346654844
>19990822       119812750       346685012
>19990823       119798906       346677122
>19990824        51920295       248842976
>19990825        65590833       324341030
>-------------------------------------------
>
>The first three days, we seemed to get all the products, the last
>two, it appears data is missing from both model sets, so I'm
>assuming it's a feed-wide problem. Can someone check on the upstream
>hosts to try and track down where the problem is? I understand
>this is not an "operational" feed, but I want to understand 
>why the data is missing sometimes.
>
>Thanks
>
>-jim
>
>-- 
>Jim Cowie                              Software Engineer
>WITI Corporation                       address@hidden
>3300 Mitchell Lane 
>Boulder, CO 80301                      (303) 497-8584
>