[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[IDD #CLZ-441499]: Noticing retransmitted? products over IDD, noaaport or IDD issue?
- Subject: [IDD #CLZ-441499]: Noticing retransmitted? products over IDD, noaaport or IDD issue?
- Date: Fri, 05 Mar 2021 08:53:53 -0700
Hi Daryl,
re:
> Over the past day or so, I have noticed a considerable number of
> "duplicate" products flowing over the IDD, of course after a considerable
> delay as LDM deduplication would squash these otherwise. I am curious
> from your noaaport ingest logs, if you can see any further details on
> this or if there is some issue with the IDD.
It is most likely that the high number of retransmissions of products is
a result of the twice per year solar interference (SI) period that
NOAAPort ingest sites are experiencing. The number of duplicates
detected and rejected by the LDMs on all of the ingest machines that
feed into the IDD have been much larger over the past three days as
the SI period has progressed, so it would not be surprising that the
numbers that were not detected and rejected would be way up also.
I would guess that the cause of the "late" duplicates is the intentional
retransmission of products in NOAAPort so that WFO sites get the
products that they missed because of their own SI caused data loss.
re:
> Here's an example, that my id3b site shows nicely.
>
> https://id3b.agron.iastate.edu/
>
> Put in "/pMWWMHX" for the Product ID.
>
> It shows this product:
>
> WHUS72 KMHX 050539
> MWWMHX
>
> URGENT - MARINE WEATHER MESSAGE
> National Weather Service Newport/Morehead City NC
> 1239 AM EST Fri Mar 5 2021
>
> Arriving once at: 2021-03-05T05:39:18Z (good) and then again at
> 2021-03-05T08:06:30Z (bad). So that ~3 hour defeats the LDM 1-hour
> queue size for most hosts on the IDD. The LDM sequence numbers are
> different.
>
> 319 for the first and 214 for the second.
>
> My feed topology to idd-dc.agron.iastate.edu is just
>
> request FSL2|WMO|NGRID|FNEXRAD ".*" iddb.unidata.ucar.edu
>
> So I am curious if you could kindly check your noaaport ingest logs
> to see if it was actually received twice.
There is no doubt that products were received multiple times, and
I believe that the biannual SI period is the cause.
re:
> Maybe the NWS has some
> bad receive site that is creating lots of retransmission requests? Shrug.
Exactly!
At the risk of providing too much information, I am including what I call
summary Gap stats for the past 7 days for all of the Unidata NOAAPort
ingest machines in the following:
burns.unidata.ucar.edu MAC=00-06-76-05-02-a9 Qsize=2100M
burns:: 20210226.213112: nGap: 5 nFrame: 4164 nG1sec: 4
nG5sec: 5 nG15sec: 5 nG1min: 5 nProds: 6414367 nDups: 13830
nPinfo: 14 nPtot: 6428197
burns:: 20210227.040737: nGap: 2 nFrame: 4 nG1sec: 2
nG5sec: 2 nG15sec: 2 nG1min: 2 nProds: 6482936 nDups: 30206
nPinfo: 0 nPtot: 6513142
burns:: 20210228.214732: nGap: 222 nFrame: 969 nG1sec: 219
nG5sec: 219 nG15sec: 220 nG1min: 220 nProds: 6553229 nDups: 26394
nPinfo: 0 nPtot: 6579623
burns:: 20210301.180050: nGap: 15015 nFrame: 189682 nG1sec: 14941
nG5sec: 14990 nG15sec: 15006 nG1min: 15013 nProds: 6725350 nDups: 83207
nPinfo: 0 nPtot: 6808557
burns:: 20210302.225104: nGap: 3600 nFrame: 904040 nG1sec: 3575
nG5sec: 3588 nG15sec: 3594 nG1min: 3597 nProds: 6556313 nDups: 157711
nPinfo: 0 nPtot: 6714024
burns:: 20210303.212917: nGap: 3112 nFrame: 1141919 nG1sec: 3095
nG5sec: 3105 nG15sec: 3109 nG1min: 3109 nProds: 6283289 nDups: 109530
nPinfo: 0 nPtot: 6392819
burns:: 20210304.180325: nGap: 3267 nFrame: 1043845 nG1sec: 3252
nG5sec: 3261 nG15sec: 3265 nG1min: 3266 nProds: 6559690 nDups: 112331
nPinfo: 0 nPtot: 6672021
desi.unidata.ucar.edu MAC=00-06-76-05-02-a9 Qsize=2100M
desi:: 20210226.223842: nGap: 9 nFrame: 19880 nG1sec: 2
nG5sec: 4 nG15sec: 5 nG1min: 5 nProds: 6412759 nDups: 13831
nPinfo: 14 nPtot: 6426590
desi:: 20210227.232320: nGap: 24 nFrame: 186 nG1sec: 2
nG5sec: 2 nG15sec: 2 nG1min: 2 nProds: 6482915 nDups: 30183
nPinfo: 0 nPtot: 6513098
desi:: 20210228.233816: nGap: 227 nFrame: 988 nG1sec: 219
nG5sec: 219 nG15sec: 220 nG1min: 220 nProds: 6553361 nDups: 26381
nPinfo: 0 nPtot: 6579742
desi:: 20210301.235447: nGap: 15032 nFrame: 189754 nG1sec: 14943
nG5sec: 14992 nG15sec: 15008 nG1min: 15016 nProds: 6725352 nDups: 83180
nPinfo: 0 nPtot: 6808532
desi:: 20210302.225104: nGap: 3605 nFrame: 904065 nG1sec: 3575
nG5sec: 3588 nG15sec: 3594 nG1min: 3597 nProds: 6556313 nDups: 157704
nPinfo: 0 nPtot: 6714017
desi:: 20210303.212917: nGap: 3119 nFrame: 1141952 nG1sec: 3095
nG5sec: 3105 nG15sec: 3109 nG1min: 3109 nProds: 6283291 nDups: 109522
nPinfo: 0 nPtot: 6392813
desi:: 20210304.225059: nGap: 3274 nFrame: 1043909 nG1sec: 3251
nG5sec: 3260 nG15sec: 3264 nG1min: 3265 nProds: 6559591 nDups: 112413
nPinfo: 0 nPtot: 6672004
jackie.unidata.ucar.edu MAC=00-06-76-05-02-a9 Qsize=2100M
jackie:: 20210219.215758: nGap: 27 nFrame: 70 nG1sec: 25
nG5sec: 25 nG15sec: 25 nG1min: 25 nProds: 6238746 nDups: 17671
nPinfo: 0 nPtot: 6256417
jackie:: 20210220.202710: nGap: 34 nFrame: 875 nG1sec: 32
nG5sec: 34 nG15sec: 34 nG1min: 34 nProds: 6107137 nDups: 17222
nPinfo: 0 nPtot: 6124359
jackie:: 20210221.000000: nGap: 0 nFrame: 0 nG1sec: 0
nG5sec: 0 nG15sec: 0 nG1min: 0 nProds: 5973069 nDups: 9884
nPinfo: 0 nPtot: 5982953
jackie:: 20210222.152239: nGap: 2 nFrame: 2 nG1sec: 1
nG5sec: 1 nG15sec: 1 nG1min: 1 nProds: 6285471 nDups: 35673
nPinfo: 0 nPtot: 6321144
jackie:: 20210223.182731: nGap: 217 nFrame: 1107 nG1sec: 210
nG5sec: 213 nG15sec: 215 nG1min: 215 nProds: 6130550 nDups: 12099
nPinfo: 0 nPtot: 6142649
jackie:: 20210224.225854: nGap: 16 nFrame: 73 nG1sec: 15
nG5sec: 15 nG15sec: 15 nG1min: 15 nProds: 6077215 nDups: 11299
nPinfo: 0 nPtot: 6088514
jackie:: 20210225.175214: nGap: 76 nFrame: 245 nG1sec: 75
nG5sec: 75 nG15sec: 75 nG1min: 75 nProds: 6135929 nDups: 17036
nPinfo: 14 nPtot: 6152965
leno.unidata.ucar.edu MAC=00-06-76-05-02-a9 Qsize=12G
leno:: 20210226.212606: nGap: 5 nFrame: 5749 nG1sec: 4
nG5sec: 4 nG15sec: 4 nG1min: 4 nProds: 6370113 nDups: 56925
nPinfo: 14 nPtot: 6427038
leno:: 20210227.040737: nGap: 2 nFrame: 4 nG1sec: 2
nG5sec: 2 nG15sec: 2 nG1min: 2 nProds: 6441722 nDups: 71417
nPinfo: 0 nPtot: 6513139
leno:: 20210228.214732: nGap: 222 nFrame: 969 nG1sec: 219
nG5sec: 219 nG15sec: 220 nG1min: 220 nProds: 6510125 nDups: 69716
nPinfo: 0 nPtot: 6579841
leno:: 20210301.180050: nGap: 15012 nFrame: 189633 nG1sec: 14938
nG5sec: 14987 nG15sec: 15003 nG1min: 15010 nProds: 6661065 nDups: 147500
nPinfo: 0 nPtot: 6808565
leno:: 20210302.225104: nGap: 3597 nFrame: 904033 nG1sec: 3572
nG5sec: 3585 nG15sec: 3591 nG1min: 3594 nProds: 6507362 nDups: 206664
nPinfo: 0 nPtot: 6714026
leno:: 20210303.212917: nGap: 3111 nFrame: 1141912 nG1sec: 3094
nG5sec: 3104 nG15sec: 3108 nG1min: 3108 nProds: 6237688 nDups: 155128
nPinfo: 0 nPtot: 6392816
leno:: 20210304.180325: nGap: 3264 nFrame: 1043838 nG1sec: 3249
nG5sec: 3258 nG15sec: 3262 nG1min: 3263 nProds: 6445749 nDups: 226277
nPinfo: 0 nPtot: 6672026
uni14.unidata.ucar.edu MAC=00-06-76-05-02-a8 Qsize=100G
uni14:: 20210226.213041: nGap: 9 nFrame: 58729 nG1sec: 4
nG5sec: 5 nG15sec: 6 nG1min: 6 nProds: 6334693 nDups: 90558
nPinfo: 14 nPtot: 6425251
uni14:: 20210227.144201: nGap: 4 nFrame: 6 nG1sec: 2
nG5sec: 2 nG15sec: 2 nG1min: 2 nProds: 6402274 nDups: 110865
nPinfo: 0 nPtot: 6513139
uni14:: 20210228.214732: nGap: 244 nFrame: 1042 nG1sec: 240
nG5sec: 241 nG15sec: 242 nG1min: 242 nProds: 6465149 nDups: 114693
nPinfo: 0 nPtot: 6579842
uni14:: 20210301.180105: nGap: 4369 nFrame: 270576 nG1sec: 4296
nG5sec: 4344 nG15sec: 4359 nG1min: 4367 nProds: 6585454 nDups: 219290
nPinfo: 0 nPtot: 6804744
uni14:: 20210302.225104: nGap: 2403 nFrame: 955039 nG1sec: 2375
nG5sec: 2390 nG15sec: 2397 nG1min: 2400 nProds: 6295925 nDups: 415889
nPinfo: 0 nPtot: 6711814
uni14:: 20210303.212917: nGap: 2834 nFrame: 1197670 nG1sec: 2810
nG5sec: 2824 nG15sec: 2830 nG1min: 2831 nProds: 6104113 nDups: 285407
nPinfo: 0 nPtot: 6389520
uni14:: 20210304.180325: nGap: 2054 nFrame: 1102815 nG1sec: 2030
nG5sec: 2046 nG15sec: 2050 nG1min: 2052 nProds: 6067830 nDups: 602277
nPinfo: 0 nPtot: 6670107
The salient things to not are:
nGap - the number of Gap messages seen in all ingest log files for the day
nProds - the number of products successfully inserted into the LDM queue for
the day
nDups - the number of products _not_ inserted into the LDM queue because there
is already products with the same MD5 signatures in the LDM queue
The nG1sec, nG5sec, nG15sec and nG1min values are counts of Gaps that
occur withing 1, 5, 15 seconds and 1 minute. They are my way of
characterizing how the Gaps are experienced. When the fraction of
Gaps are received withing 1 second, the "noise" (SI in this case) is
very bursty, etc.
The other thing listed for each machine is the size of its LDM queue.
A quick comparison of the nDups vs LDM queue size shows how the number
of duplicates detected and rejected is a positive function of the
size of the LDM queue on the ingest machine - the larger the LDM queue
size, the more duplicates that are detected and rejected.
Cheers,
Tom
--
****************************************************************************
Unidata User Support UCAR Unidata Program
(303) 497-8642 P.O. Box 3000
address@hidden Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage http://www.unidata.ucar.edu
****************************************************************************
Ticket Details
===================
Ticket ID: CLZ-441499
Department: Support NOAAPORT
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata
inquiry tracking system and then made publicly available through the web. If
you do not want to have your interactions made available in this way, you must
let us know in each email you send to us.
