Unidata - To provide the data services, tools, and cyberinfrastructure leadership that advance Earth system science, enhance educational opportunities, and broaden participation. Unidata
         
  advanced  
 

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #OQD-608743]: ldm failover question



Sue,

> Can you tell me how the downstream
> LDM makes the determination of which upstream host is PRIMARY
> and which is ALTERNATE ?   Does this determination happen
> on a product by product basis ? Is it traceable by the LDM user?

A downstream LDM process keeps track of its success in adding data-products to 
the product-queue and also knows how many other downstream LDM peer-processes 
have requested the same data.

A downstream LDM process in alternate mode will disconnect and reconnect in 
primary mode if and only if it has been more successful in adding data-products 
to the product-queue than any other peer-process over the last minute.

Similarly, a downstream LDM process in primary mode will disconnect and 
reconnect in alternate mode if and only if it has not been more successful in 
adding data-products to the product-queue than any other peer-process over the 
last minute.

Downstream LDM processes log when they switch modes.

The rtstats(1) web-pages can be used to determine the relative efficiency of 
routes.

> I am unclear on what it means to "ensure that the data-product signatures
> (i.e., MD5 checksums) [are] computed identically. "
> How does the LDM user do this?

The product-signature is computed by the LDM ingester responsible for creating 
the data-product and adding it to the product-queue.

> Certainly the script which creates product ids, feed-type,
> and the insert command are indentical on upstream hosts.

If the data-products are inserted using the pqinsert(1) utility in identical 
ways, then the product-signatures will be identical.

> It is possible that the underlying data for a particular
> product id  could differ if one host is experiencing
> some data latency issues that effect product generation.
> Is that an issue that could cause duplicate products( product ids)
> in the queue?

The only way I can think of that this would happen is if the pqinsert(1) 
utility was told to use the product-identifier rather than the product data as 
the basis for the product-signature and the identifier encoded the time.

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: OQD-608743
Department: Support LDM
Priority: Normal
Status: Closed


 
 
  Contact Us     Site Map     Search     Terms and Conditions     Privacy Policy     Participation Policy
 
National Science Foundation (NSF) UCAR Community Programs   Unidata is a member of the UCAR Community Programs, is managed by the University Corporation for Atmospheric Research, and is sponsored by the National Science Foundation.
P.O. Box 3000     Boulder, CO 80307-3000 USA     Tel: 303-497-8643     Fax: 303-497-8690