[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #XYU-166657]: data duplication

Hi Karen,

> Brief background -- We use ldm to pass data between machines here at
> NSSL for algorithm processing.  For example we get radar data, perform
> QC on it and pass it to another machine via LDM where it gets further
> processed.  Now, the developers want to do comparisons between different
> QC techniques, and send them on downstream for further processing.
> Unfortunately sometimes the different QC methods produce identical
> results, and one or the other product (which have different feedtypes
> and paths for downstream differentiation) doesn't get put into the queue
> because the MD5 checksum is the same.
> 
> I realize there is no way to put duplicate files in the queue (I've
> searched the ldm support threads at Unidata and I can see that), but I'm
> wondering if I can use multiple queues and put the different methods
> product into separate queues.  I could swear I've seen reference to
> using multiple queues, but I can't seem to find anything now with all my
> searching, and I'm not sure just how to set it up.
> 
> I've also asked the developers if they can just put a flag or small mod
> in the file so that it won't be duplicated, but I'm not sure if they
> will bite if there is another way to do this.

It's possible to run multiple LDM-s on the same host.  See
<http://www.unidata.ucar.edu/software/ldm/ldm-current/basics/multiple_ldms.html>.

Step 4 in the above-mentioned web page is incomplete.  It should, instead,
say

4. Decide on one of the following approaches for allowing connections
from downstream sites:

    A.  Decide on a distinct IP address to use. This necessitates the
        use of a distinct network interface, which could be a real one
        (implemented in hardware) or a virtual one (implemented in software).
        When configuring the ldmadmin(1)  configuration-file,
        etc/ldmadmin-pl.conf, use the $ip_addr variable to specify that IP
        address.

    B.  Select a distinct port on which the LDM server will listen.  Use
        the "--enable-port=port" option of the configure script to
        configure the LDM to use that port.  On the downstream hosts, do
        one of the following:

            1.  Use the optional ":port" field in REQUEST entries to 
                specify to which port to connect.

            2.  Configure the downstream LDM to use the same port via
                the same "--enable-port=port" option of the 
                configure script.

Good luck.

> -------------------------------------------
> 
> There are 2 kinds of people in the world:
> 
> 1) Those who can extrapolate from incomplete data.
> 
> -------------------------------------------
> address@hidden
> 
> Phone:  405-325-6982
> Cell: 405-834-8559
> SAIC/Systems Analyst
> National Severe Storms Laboratory

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: XYU-166657
Department: Support LDM
Priority: Normal
Status: Closed