[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Missing products



Harry Edmon wrote:
> 
> Using the notifyme output I already had, I looked at a disconnect
> event with thelma.  It does appear that when the reconnect occurs, it
> makes a request based on the time stamp of the last product received
> (call that product A). I can even see that thelma sends product A
> again.  BUT, by looking at the time stamps in the notifyme
> output, you can see that thelma does not send out products with
> monotonically increasing time stamps - i.e. it received products out of
> order.  So in my disconnect event, any product received by thelma after
> product A, with at time stamp earlier than product A, will never be
> sent to me.
> 
> This does not explain everything I am seeing, but it is a first step.
> --
> Dr. Harry Edmon                 E-MAIL: address@hidden
> 206-543-0547                            address@hidden
> Dept of Atmospheric Sciences    FAX:    206-543-0308
> University of Washington, Box 351640, Seattle, WA 98195-1640


Right - this is exactly the scenario we considered.  We know that thelma
recieves products out of order.  Because of this, in our meeting the day
you were here we discussed modifying the queue algorithms to order
products by origination time instead of local ingest time.  Also, I'm
investigating the possibility of adding the option of going back in time
for N seconds after a disconnect, at the expense of possibly receiving
duplicates.  But, this last option would not address the real problem -
you could still miss products if you don't go back "far enough". 
Another problem with this approach is that if you are already behind and
the you lose a connection, going back in time could put you even further
behind, causing a cyle of increasing behindness.  

We're postponing decisions on these issues until Russ can complete some
research on the issue of the apparent high cost of reestablishing the
place in the queue after a disconnect.  That seems most important and
could be a significant bug.

Anne
-- 
***************************************************
Anne Wilson                     UCAR Unidata Program            
address@hidden                 P.O. Box 3000
                                  Boulder, CO  80307
----------------------------------------------------
Unidata WWW server       http://www.unidata.ucar.edu/
****************************************************