Re: Pattern in Late Data

>For the last few weeks, I've been noticing a pattern in late or missing 
>data here at SIO.  Between about 02Z and 05Z my product queue frequently 
>gets well behind the power curve.  The METARs (and, probably, every other 
>obs type) can be as much as 90 minutes late getting into the hourly files.
>Most of the time the system is able to handle this acceptably but, on 
>occasion, many of the observations are lost.  This occurred again last 
>night.  My 20021009 03Z METAR file, which should be around 550,000 bytes, 
>is only 211,206 bytes.  All of today's files from 02Z (possibly as early as 
>01Z) through 04Z are significantly smaller than I would normally expect.
>This is probably not a CPU usage problem as the IDD machine 
>( has no other function in life.  All it does is process 
>the LDM and build some text files for our weather web page.
>Is this problem unique to SIO (and my downstream nodes)?  If so, I'll look 
>for a solution locally.  Is anybody else experiencing delays like 
>this?  Does this have anything to do with the 00Z HDS product cycle?  If 
>this problem isn't unique to the SIO branch of the IDD tree, then I'll buck 
>this up to the Unidata support people.


I have about a weeks work of processed surface data on my systems.  I see no
indications of any time of problems anywhere.  I don't see any missing data.
Everything looks normal here.

You might want to monitor your IDD machine tonight to see if you can get any
ideas about what is happening.

        Kevin W. Thomas
        Center for Analysis and Prediction of Storms
        University of Oklahoma
        Norman, Oklahoma
        Email:  kwthomas@xxxxxx