[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #XYU-166657]: data duplication



Hi Karen,

> Brief background -- We use ldm to pass data between machines here at
> NSSL for algorithm processing.  For example we get radar data, perform
> QC on it and pass it to another machine via LDM where it gets further
> processed.  Now, the developers want to do comparisons between different
> QC techniques, and send them on downstream for further processing.
> Unfortunately sometimes the different QC methods produce identical
> results, and one or the other product (which have different feedtypes
> and paths for downstream differentiation) doesn't get put into the queue
> because the MD5 checksum is the same.
> 
> I realize there is no way to put duplicate files in the queue (I've
> searched the ldm support threads at Unidata and I can see that), but I'm
> wondering if I can use multiple queues and put the different methods
> product into separate queues.  I could swear I've seen reference to
> using multiple queues, but I can't seem to find anything now with all my
> searching, and I'm not sure just how to set it up.
> 
> I've also asked the developers if they can just put a flag or small mod
> in the file so that it won't be duplicated, but I'm not sure if they
> will bite if there is another way to do this.

It's possible to run multiple LDM-s on the same host.  See
<http://www.unidata.ucar.edu/software/ldm/ldm-current/basics/multiple_ldms.html>.

Step 4 in the above-mentioned web page is incomplete.  It should, instead,
say

4. Decide on one of the following approaches for allowing connections
from downstream sites:

    A.  Decide on a distinct IP address to use. This necessitates the
        use of a distinct network interface, which could be a real one
        (implemented in hardware) or a virtual one (implemented in software).
        When configuring the ldmadmin(1)  configuration-file,
        etc/ldmadmin-pl.conf, use the $ip_addr variable to specify that IP
        address.

    B.  Select a distinct port on which the LDM server will listen.  Use
        the "--enable-port=port" option of the configure script to
        configure the LDM to use that port.  On the downstream hosts, do
        one of the following:

            1.  Use the optional ":port" field in REQUEST entries to 
                specify to which port to connect.

            2.  Configure the downstream LDM to use the same port via
                the same "--enable-port=port" option of the 
                configure script.

Good luck.

> -------------------------------------------
> 
> There are 2 kinds of people in the world:
> 
> 1) Those who can extrapolate from incomplete data.
> 
> -------------------------------------------
> address@hidden
> 
> Phone:  405-325-6982
> Cell: 405-834-8559
> SAIC/Systems Analyst
> National Severe Storms Laboratory

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: XYU-166657
Department: Support LDM
Priority: Normal
Status: Closed


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.