Re: [ldm-users] Question about duplicate product checking

  • To: Steven Emmerson <emmerson@xxxxxxxx>
  • Subject: Re: [ldm-users] Question about duplicate product checking
  • From: Matthew Foster - NOAA Federal <matthew.foster@xxxxxxxx>
  • Date: Fri, 6 Apr 2018 11:25:43 -0500
Steve,

Actually, the combination of both responses was very enlightening.  Thank
you for your insight!

Matt


On Thu, Apr 5, 2018 at 11:01 AM, Steven Emmerson <emmerson@xxxxxxxx> wrote:

> Matt,
>
> Let me amend my previous reply. I answered your question correctly, but
> from too-narrow a perspective (one of the risks of getting an answer from
> the developer I suppose 😉).
>
> If products are being sent over a connection that's in the alternate
> transfer mode (as opposed to primary transfer mode) then, initially, only
> the metadata of the product is sent. The downstream LDM then calls a
> product-queue function that determines whether or not the product is
> already in the queue. If it is, then the downstream LDM responds with a
> "don't send" message to the upstream LDM and the product isn't sent. If the
> product is not in the queue, however, then the downstream LDM responds with
> a "send" message. When the product is then received by the downstream LDM,
> the downstream LDM again checks to see if the product is already in the
> queue (in case another process added it) and rejects the insertion if it
> is. The result of all this checking is reduced traffic on the alternate
> transfer mode connection.
>
> Is that what you wanted to know?
>
> Regards,
> Steve Emmerson
>
> On Thu, Apr 5, 2018 at 9:38 AM, Steven Emmerson <emmerson@xxxxxxxx> wrote:
>
>> The functions in the product-queue component that are responsible for
>> inserting a data-product determine whether or not the product already
>> exists in the product-queue and reject the insertion if it does.
>>
>> Regards,
>> Steve Emmerson
>>
>> On Thu, Apr 5, 2018 at 9:25 AM, Matthew Foster - NOAA Federal <
>> matthew.foster@xxxxxxxx> wrote:
>>
>>> I was trying to find the answer to this question in the online
>>> documentation, but I couldn't find it clearly stated.
>>>
>>> When, exactly, does duplicate checking occur? Does the downstream LDM
>>> receive the data product *then* do the duplicate check before inserting
>>> into the queue, OR does the upstream send the product's checksum and the
>>> accept/reject decision is made at that point (i.e. before the product is
>>> sent)? I've always assumed it was the latter, but now I'm not 100% sure .
>>>
>>> Thanks!
>>>
>>> Matt
>>>
>>>
>>> --
>>> Matt Foster, Science & Operations Officer
>>> NWS Operations Proving Ground
>>> 7220 NW 101st Terrace
>>> <https://maps.google.com/?q=7220+NW+101st+Terrace+Kansas+City,+MO+64153+Office:+816&entry=gmail&source=g>
>>> Kansas City, MO 64153
>>> <https://maps.google.com/?q=7220+NW+101st+Terrace+Kansas+City,+MO+64153+Office:+816&entry=gmail&source=g>
>>> Office: 816
>>> <https://maps.google.com/?q=7220+NW+101st+Terrace+Kansas+City,+MO+64153+Office:+816&entry=gmail&source=g>
>>> -994-3072
>>>
>>>
>>>
>>> _______________________________________________
>>> NOTE: All exchanges posted to Unidata maintained email lists are
>>> recorded in the Unidata inquiry tracking system and made publicly
>>> available through the web.  Users who post to any of the lists we
>>> maintain are reminded to remove any personal information that they
>>> do not want to be made public.
>>>
>>>
>>> ldm-users mailing list
>>> ldm-users@xxxxxxxxxxxxxxxx
>>> For list information or to unsubscribe,  visit:
>>> http://www.unidata.ucar.edu/mailing_lists/
>>>
>>
>>
>


-- 
Matt Foster, Science & Operations Officer
NWS Operations Proving Ground
7220 NW 101st Terrace
Kansas City, MO 64153
Office: 816-994-3072
  • 2018 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: