[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #ZZW-504319]: LDM catching up



Carl,

> I've noticed that since we've upgraded from the 6.4.x to 6.5.1, LDM now
> does it's catch up differently.  For example, if the LDM server goes off
> the network for an hour and then comes back online, the client has to
> catch up to the latest data in the queue.  In 6.4.x, the oldest file in
> the queue (not yet received by the client) would be sent to the client
> first; once that file was complete the next file in line would be sent
> and so-on until the queue was sending current data.  In 6.5.1, when the
> server comes back online, the client appears to receive several files
> all at once (in parallel) but if there are bandwidth limitations, then
> the files trickle in a little bit at a time.  The problem with the
> parallel method is this: a partial file cannot be fully processed until
> it arrives; also, we've seen newer files arrive completed before older
> files.  When algorithms are dependent on data that arrives in a
> time-sequence, this can cause problems.
> 
> My question is: can we turn this "feature" off?  We'd like to have the
> files arrive one at a time in the proper time-sequence...

I suggest upgrading to LDM 6.6.3.  A new feature of 6.6 is the
addition of a persistent "state" file for each downstream
LDM.  The purpose of the file is to contain information
on the last, successfully-received data-product for that
particular request.

By the way, if an LDM receives the same data-products via
multiple REQUEST entries to different upstream sites, then
there's no way to guarantee that the data-products are
received in the order in which they're created at the
source site.

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: ZZW-504319
Department: Support LDM
Priority: Normal
Status: Closed


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.