Re: [ldm-users] question about primary and secondary data transfer

Greg

Gilbert is correct. LDM is trying to get the fastest site. Once upon a
time, we could identify primary and secondary upstream sites but those days
are now gone.

There might be a way to define your requests that's better than 'REQUEST
ANY ".*" ', but in general terms, in order to have the redundancy, I don't
think it would reduce your network traffic.

That said, if your NOAAPort ingest server is on the local network and
you're collecting data from an off-site downstream, your overall bandwidth
offsite shouldn't be a killer, necessarily. It's more a nuisance that it's
going back and forth between servers.

Gerry

On Sat, Jan 13, 2018 at 12:00 AM, Gilbert Sebenste <gilbert@xxxxxxxxxxxxxxxx
> wrote:

> Hi Greg,
>
> It will also try to pick the fastest site. If both are pretty much equal,
> if I recall correctly, it will flip back-and-forth between those two
> machines. Hopefully somebody can correct me here if I’m wrong.
>
> Gilbert
>
> On Jan 12, 2018, at 10:43 PM, Karen Cooper - NOAA Affiliate <
> karen.cooper@xxxxxxxx> wrote:
>
> Hi Greg,
>
> I have a feeling it's trying to get data from both sources and trying to
> balance between them.
>
> It would be nice if you could select a preferred source and a secondary
> source that you only go to if your preferred source goes down for a
> specified amount of time.
>
> On Fri, Jan 12, 2018 at 1:43 PM, Greg Trotter <gtrotter@xxxxxxxxxx> wrote:
>
>> I have an LDM server that is connected to two upstreams – one is my
>> NOAAPORT ingester on my local network, and one is a backup site at a remote
>> location. I have both listed in my ldmd.conf, with the local server listed
>> first, and the request patterns are identical.
>>
>>
>>
>> Looking at the queue, it looks like all is well – the origin on all the
>> products is from our local NOAAPORT ingest server. If I look at the network
>> traffic on that server, however, I see a lot of data coming in from the
>> remote server – almost as much as we are bringing in from the local server.
>> I theorize that the downstream doesn’t see it locally, and requests it from
>> the remote upstream, but by the time it arrives, it’s come in from the
>> local upstream? I can’t find any other reason that we’d be having that much
>> traffic from an upstream LDM that never appears as an origin for the
>> products in our queue.
>>
>>
>>
>> Is there something I am doing wrong? Clearly, I’d prefer not to use that
>> much bandwidth if we don’t have to. Is there a way to have it only hit the
>> remote upstream if the local is unavailable?
>>
>>
>>
>> Here’s the relevant part of my ldmd.conf:
>>
>>
>>
>>
>>
>> REQUEST ANY     ".*"    <IP of local noaaport ingester>
>>
>> REQUEST ANY     ".*"    <IP of remote noaaport ingester>
>>
>>
>>
>> *Greg Trotter*
>>
>> System Administrator
>>
>> Weather Decision Technologies, Inc
>>
>> 201 David L Boren Blvd Suite 270
>>
>> Norman, Oklahoma 73072
>>
>>
>>
>> _______________________________________________
>> NOTE: All exchanges posted to Unidata maintained email lists are
>> recorded in the Unidata inquiry tracking system and made publicly
>> available through the web.  Users who post to any of the lists we
>> maintain are reminded to remove any personal information that they
>> do not want to be made public.
>>
>>
>> ldm-users mailing list
>> ldm-users@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe,  visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>>
>
>
>
> --
> “The presence of those seeking the truth is infinitely to be preferred to
> the presence of those who think they’ve found it."
>  -- Terry Prachett
>
> -------------------------------------------
> Karen.Cooper@xxxxxxxx
>
> Phone#:  405-325-6456 <(405)%20325-6456>
> Cell:   405-834-8559 <(405)%20834-8559>
> National Severe Storms Laboratory
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web.  Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/
>
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web.  Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/
>



-- 
Gerry Creager
NSSL/CIMMS
405.325.6371
++++++++++++++++++++++
“Big whorls have little whorls,
That feed on their velocity;
And little whorls have lesser whorls,
And so on to viscosity.”
Lewis Fry Richardson (1881-1953)
  • 2018 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: