[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #KAM-474743]: LDM catching up



Carl,

> I've noticed that since we've upgraded from the 6.4.x to 6.5.1, LDM now
> does it's catch up differently.  For example, if the LDM server goes off
> the network for an hour and then comes back online, the client has to
> catch up to the latest data in the queue.  In 6.4.x, the oldest file in
> the queue (not yet received by the client) would be sent to the client
> first; once that file was complete the next file in line would be sent
> and so-on until the queue was sending current data.  In 6.5.1, when the
> server comes back online, the client appears to receive several files
> all at once (in parallel) but if there are bandwidth limitations, then
> the files trickle in a little bit at a time.  The problem with the
> parallel method is this: a partial file cannot be fully processed until
> it arrives; also, we've seen newer files arrive completed before older
> files.  When algorithms are dependent on data that arrives in a
> time-sequence, this can cause problems.
> 
> My question is: can we turn this "feature" off?  We'd like to have the
> files arrive one at a time in the proper time-sequence...

An upstream LDM 6.5 process uses a different algorithm
than an upstream LDM 6.4 process to decide when to flush
the connection to a downstream LDM process.  The algorithm
can result in a burst of data-products.  The products will
still be in the same order, however.  The algorithm was
further modified in LDM 6.6.3 to make the flushing less
bursty.  I suggest that you upgrade to that version.

In all versions of the LDM, data-products are sent in
the order in which they were inserted into the product-
queue.  It's possible that you are receiving the same
data-products via more than one upstream site.  This 
kind of multiple-path delivery can result in data-
products that appear to be received out of order.

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: KAM-474743
Department: Support LDM
Priority: Urgent
Status: Closed


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.