Pattern in Late Data

For the last few weeks, I've been noticing a pattern in late or missing data here at SIO. Between about 02Z and 05Z my product queue frequently gets well behind the power curve. The METARs (and, probably, every other obs type) can be as much as 90 minutes late getting into the hourly files.

Most of the time the system is able to handle this acceptably but, on occasion, many of the observations are lost. This occurred again last night. My 20021009 03Z METAR file, which should be around 550,000 bytes, is only 211,206 bytes. All of today's files from 02Z (possibly as early as 01Z) through 04Z are significantly smaller than I would normally expect.

This is probably not a CPU usage problem as the IDD machine ( has no other function in life. All it does is process the LDM and build some text files for our weather web page.

Is this problem unique to SIO (and my downstream nodes)? If so, I'll look for a solution locally. Is anybody else experiencing delays like this? Does this have anything to do with the 00Z HDS product cycle? If this problem isn't unique to the SIO branch of the IDD tree, then I'll buck this up to the Unidata support people.


-----===(* Climate's what we expect, but weather's what we get. *)===----- Larry Riddle : Climate Research Division : Scripps Institution of Oceanography
      University of California, San Diego : La Jolla, California  92093-0224
      Phone: (858) 534-1869 : Fax: (858) 534-8561 : E-Mail: lriddle@xxxxxxxx