[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: losing products


Sorry to have taken so long to answer this ...

> I am still trying to figure out why we are missing some grib messages.
> We save all the thin AVN files into a raw directory and then run a
> check to see if they are all there before we do some processing.  We
> consistently are missing 10-20 messages.

We've discovered an interaction between the SDI ingest system and our
pqing code run at the ingest sites that results in some dropped
products.  For example, in the last hour, 3 GRIB products were dropped
because of this combination of bugs.  We're close to having a
workaround and a more robust ingest solution that won't have this
problem, but for now, the only "solution" is to get a redundant feed
from the wunderground.com ingester (you'd have to ask Jeff Masters for
exactly which host), where they're running an older version of the SDI
code that doesn't exhibit this particular problem.  That's what we're
doing in our top-level ring, so motherlode gets a few of its products
from wunderground and the rest from the other two top-level ingest

> I have my queue set to 975MB (full NOAAPORT SDI feed including
> NEXRAD).  I noticed I just got a nfree of 0 while running pqmon.  Is
> that a sign of a problem?
> Aug 02 14:02:24 pqmon: Starting Up (877)
> Aug 02 14:02:24 pqmon: nprods nfree  nempty      nbytes  maxprods  maxfree
> minempty    maxext  age
> Aug 02 14:02:24 pqmon: 165379     1   72657   975003208    179024        8
> 59012       440 16397
> Aug 02 14:02:39 pqmon: 165442     0   72595   975003648    179024        8
> 59012         0 16395

No, nfree just goes to zero occasionally when the last free region in
the product queue has just been allocated and no more free regions are
available.  When the next product comes in, it will request a free
region and the queue code will delete the oldest products in the queue
until it can find one large enough, which will replenish the free
region list and cause nfree to increase to a larger value again.