[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

19990716: CONDUIT latencies

Dave,

There are some things we can do to tune the LDM requests,
and we can look at other upstreams. 

There is currently 1 bottle neck that drives the initial latencies up.
This is the T-1 line from NCEP to GSFC which is roughly capable of
delivering 550MB per hour. Since the AVN (now out to 84 hours) can exceed
600MB, and the MRF is also pushing 500MB then things can get backlogged-
especially when NCEP has Cray crashes and then come back up and post
the data all at once.

I have found I can get better throughput to the upstream using
seaparte request lines like:

request NMC2    ".*[036]$"       128.183.252.98
request NMC2    ".*[147]$"       nora.gsfc.nasa.gov
request NMC2    ".*[^013467]$"   conduit_a

I provided an alias in our hosts table for the third entry above.
By using different names, the LDM actually will create a seperate
rpc.ldmd connection for each subset.

The idea here is that all the conduit products have a sequence number
tagged on to the end, and the split requests seem to get more
round robin time from the router than a single request.

Unfortunately PSU did not feel like they had a good connection to gsfc,
so the feed from back here same as UIUC, but I was wondering what your
traceroutes to profhorn.meteor.wisc.edu and nora.gsfc.nasa.gov look like.

For the past month we have also been downgraded from out 100baseT
connection to a 10baseT connection while our building is rewired.
When our switch gets moved to the new wiring closet it could improve
things as well.


Steve Chiswell
Unidata User Support



On Fri, 16 Jul 1999, David J. Knight wrote:

> Hi,
>      We've being receiving the CONDUIT/NMC2 data feed
> at UAlbany for some time now. Unfortunately it has not proven
> to be particularly reliable. Specifically, there are
> many hours of the day when latencies get to 3600 sec
> and we start losing data. Right now the nmc2 feed is
> of little value to us, so we have to continue to
> ftp the files from the oso server.
> 
>      I wonder if other NMC2 sites are seeing *very*
> inconsistant receipt of data? If not, then it is 
> probably a local problem, so you don't need to read
> further unless you are willing to serve us the feed ;)
> 
> We currently obtain the feed from flood.atmos.uiuc.edu
> Here are entries from last 24 hours of stat files:
> 
> 1999071512.stats:19990715125444    NMC2                       tgsv7a       
> 8430  121583537    1658.38 3660@5442 19990715135542
> 1999071513.stats:19990715134129    NMC2                       tgsv7a         
> 11       1225    3053.57 3640@2725 19990715143959
> 1999071514.stats:19990715145703    NMC2                       tgsv7a       
> 4303  105105886    3256.68 3661@3443 19990715155125
> 1999071515.stats:19990715155934    NMC2                       tgsv7a       
> 1483   32642622    3224.65 3660@1245 19990715162745
> 1999071516.stats:19990715165234    NMC2                       tgsv7a         
> 85       9179    2657.02 3560@4717 19990715174638
> 1999071518.stats:19990715184725    NMC2                       tgsv7a         
> 15       1675     163.28  516@0710 19990715184845
> 1999071519.stats:19990715195817    NMC2                       tgsv7a       
> 4484  109037162    1405.73 3656@5814 19990715205910
> 1999071520.stats:19990715203610    NMC2                       tgsv7a       
> 3645   60369581    3017.24 3660@0115 19990715213615
> 1999071521.stats:19990715215229    NMC2                       tgsv7a         
> 21       2345     465.78 1006@5221 19990715220907
> 1999071522.stats:19990715224341    NMC2                       tgsv7a         
> 12       1340     406.33  923@4327 19990715225849
> 1999071523.stats:19990715234338    NMC2                       tgsv7a         
> 12       1340     340.78  876@4326 19990715235802
> 1999071600.stats:19990716003757    NMC2                       tgsv7a          
> 3        335     134.08  271@3757 19990716004227
> 1999071601.stats:19990716015253    NMC2                       tgsv7a         
> 33       3685    1091.47 2428@5233 19990716023301
> 1999071602.stats:19990716025911    NMC2                       tgsv7a       
> 2453   59397579    2824.92 3660@2035 19990716040007
> 1999071603.stats:19990716035113    NMC2                       tgsv7a        
> 800   18925585    3577.05 3660@0206 19990716045200
> 1999071604.stats:19990716045539    NMC2                       tgsv7a          
> 8        892    3210.54 3545@5539 19990716055445
> 1999071605.stats:19990716054700    NMC2                       tgsv7a         
> 12       1340    1517.95 2727@4651 19990716063217
> 1999071606.stats:19990716064901    NMC2                       tgsv7a       
> 3080   74064891    2620.21 3660@2430 19990716074938
> 1999071607.stats:19990716075937    NMC2                       tgsv7a       
> 1932  120454025    3094.27 3661@5311 19990716090037
> 1999071608.stats:19990716083903    NMC2                       tgsv7a       
> 2320   46350502    3575.35 3661@0515 19990716093802
> 1999071609.stats:19990716095420    NMC2                       tgsv7a         
> 19       2120    1531.36 3359@5420 19990716105019
> 1999071610.stats:19990716104532    NMC2                       tgsv7a         
> 11       1228    2111.46 3260@4532 19990716113952
> 1999071611.stats:19990716114554    NMC2                       tgsv7a          
> 9       1003    1538.50 2273@4554 19990716122347
> 
> Looking at 
> http://www.unidata.ucar.edu/staff/chiz/ncep_idd/gifs/latency_NMC.gif
> it appears this is the worse possible place to obtain the feed from!
> 
> 
> I wonder if would be possible to get the feed from some other site.
> Perhaps PSU. traceroutes look good to there (at least right now).
> 
> traceroute ihlseng.ugems.psu.edu
> traceroute to ihlseng.ugems.psu.edu (128.118.41.50), 30 hops max, 40 byte 
> packets
>  1  be102 (169.226.4.1)  2 ms  1 ms  1 ms
>  2  169.226.13.1 (169.226.13.1)  1 ms  1 ms  1 ms
>  3  at-gw3-alb-1-0-T3.appliedtheory.net (169.130.23.5)  2 ms  9 ms  2 ms
>  4  at-gw2-alb-0-0.appliedtheory.net (169.130.20.2)  3 ms  3 ms  3 ms
>  5  at-gw2-syr-1-1-0-T3.appliedtheory.net (169.130.1.22)  6 ms  4 ms  4 ms
>  6  at-bb1-pen-5-0-0-T3.appliedtheory.net (169.130.1.121)  17 ms  18 ms  18 ms
>  7  sl-bb11-pen-4-2.sprintlink.net (144.232.5.113)  18 ms  17 ms  17 ms
>  8  sl-bb12-pen-10-0.sprintlink.net (144.232.5.158)  17 ms  16 ms  16 ms
>  9  sl-bb10-rly-10-1.sprintlink.net (144.232.8.153)  17 ms  16 ms  17 ms
> 10  144.232.0.62 (144.232.0.62)  18 ms  18 ms  17 ms
> 11  sl-pittssup-1-0-0-T3.sprintlink.net (144.232.184.10)  28 ms  30 ms  26 ms
> 12  penn-state.psc.net (192.88.115.90)  30 ms  30 ms  30 ms
> 13  Willard1-Fddi5-0.gw.psu.edu (128.118.100.4)  29 ms  31 ms  30 ms
> 14  172.28.20.2 (172.28.20.2)  32 ms  31 ms  30 ms
> 15  ihlseng.ugems.psu.edu (128.118.41.50)  30 ms *  31 ms
> 
> Actually traceroutes to flood.atmos.uiuc.edu don't look too bad either
> ...
> 15  flood.atmos.uiuc.edu (128.174.80.47)  44 ms  58 ms  48 ms
> 
> But they get the data later than most sites, and by the time
> it gets to us much of it gets dropped
> 
> Thanks for your consideration
> David
> 
> David Knight
> Department of Earth and Atmospheric Sciences   Tel: (518)-442-4204
> SUNYA   ES-228                                 Fax: (518)-442-4494
> Albany, NY  12222                              Email: address@hidden
> 
> 
>