I tried to send this earlier, but it got bounced from the list... so I'll
try it again.
We are seeing the same thing here at Texas Tech and I have been fighting
the problem for the last six months. While I have not been able to rectify
the situation, I think I have isolated the problem.
For about 20 hours out of the day, we see 60+ minute latentcies for
IDS|DDPLUS and HDS, and accceptable latencies in NLDN, PCWS, FSL2, and
MCIDAS. Between 4 AM and 8 AM local time all latencies are great (~ 5
sec). Through dealing with the campus network folks, we have figured out
that this is the time when our dorm internet usage finally falls off. At
all other times, the campus line is saturated with student use.
To trouble shoot this I have spent a lot of time trying to identify the
quality of upstream connections, upgraded our ldm ingestor to a SPARC based
machine instead of an X86, and split the data feeds as is often suggested.
This worked a little bit, but still leaves us missing data for most of the
day. I have also tried turning the HDS requests off completely... and this
had no effect. Nor did splitting the feed so that HDS was separated from
the other feeds.
As more dorms are wired, this problem will likely get worse. I have
complained loudly about this, but unfortunately keeping the students happy
is more important than facilitating our data needs. I have also been told
that short of upgrading our network connection, there is little that can be
done to reduce these latencies.
Anyway, I can't tell you why the MCIDAS feed (or the FSL2 feed for that
matter) is different, but we do witness the same effect. If you come up
with an answer, I'd be glad to hear it.
-Tim
At 12:35 PM 11/15/00 -0500, Jim Koermer wrote:
>Tom et al.,
>
>During these episodes, my upstream site(s) is usually quite good with
>FOS latencies < 1 minute.
>
>I am thinking that the HDS feed is the culprit. Does anyone know if I
>turn my HDS request off whether this will affect my downstream feed's
>ability to get HDS? I think the answer is "yes".
>
>Perhaps, I need to try splitting the IDD ingestion on two machines--one
>for HDS and the other for DDPLUS.
>
> Jim
>
>Tom McDermott wrote:
>>
>> Jim,
>>
>> I'm probably not very qualified to answer your questions, but I'll give it
>> a try based on my own understanding. I'm sure more knowledgeable people
>> can offer corrections.
>>
>> On Tue, 14 Nov 2000, Jim Koermer wrote:
>>
>> > It seems that lately everytime that I check the Unidata topology pages,
>> > usually after I am seeing data problems, I seem to see the same thing--
>> > ~+60 minute latencies on FOS, but only a few minutes on McIDAS. Why is
>> > there such a large difference in latencies, since I'm getting data from
>> > the same upstream site (UIUC or UMICH - it doesn't seem to matter which
>> > one)?
>>
>> You may be getting it from the same upstream site, but the top level sites
>> are getting it from different sources. The top level site for MCIDAS is
>> at SSEC, whereas the toplevel site for WMO, motherlode, is at Unidata,
>> which in turn is fed by 3 ingesting hosts at unidata, weather underground
>> and SSEC. Now the amount of data on the WMO feed is much greater than the
>> MCIDAS feed, and this can result in greater latencies, especially when the
>> gridded data is coming in (this is especially true on the days they
>> transmit the models twice). The FOS is really NOAAPORT now, whereas I
>> believe MCIDAS is not part of NOAAPORT, so this could account for some of
>> the discrepancy.
>>
>> One thing you don't mention is whether your upstream site is also showing
>> +60 minute latencies on FOS. If not, the problem may be in the quality of
>> your network connection to your upstream host. I suppose you could ask
>> Unidata for alternative upstream hosts if the problem persists. This may
>> or may not help, depending on whether your network bandwidth is maxed out
>> our not.
>>
>> > It also seems that a number of downstream sites are in the ~+60 minute
>> > on FOS, but much lower on McIDAS. I've noticed this many times
>> > recently. At 2250Z, I still haven't received any hourly ob data for
>> > 22Z, yet I'm sure that if I check back in an hour, I'll see most of the
>> > obs eventually came in. Why should the obs that come in small batches
>> > be so delayed, yet the much larger McIDAS areafiles are coming in fine?
>> >
>>
>> Generally most sites request the text data on IDS|DDPLUS over the same
>> connection as the HDS feed, so if there is a slowdown with HDS, IDS|DDPLUS
>> will be affected as well.
>>
>> Tom
>>
------------------------------------------------------------------------------
>> Tom McDermott Email:
tmcderm@xxxxxxxxxxxxxxxxxxxxx
>> System Administrator Phone: (716) 395-5718
>> Earth Sciences Dept. Fax: (716) 395-2416
>> SUNY College at Brockport
>>
>> > Something just doesn't seem right.
>> >
>> > Jim
>> > --
>> > James P. Koermer E-Mail: koermer@xxxxxxxxxxxxxxxxx
>> > Professor of Meteorology Office Phone: (603)535-2574
>> > Natural Science Department Office Fax: (603)535-2723
>> > Plymouth State College WWW: http://vortex.plymouth.edu/
>> > Plymouth, NH 03264
>> >
>
>--
>James P. Koermer E-Mail: koermer@xxxxxxxxxxxxxxxxx
>Professor of Meteorology Office Phone: (603)535-2574
>Natural Science Department Office Fax: (603)535-2723
>Plymouth State College WWW: http://vortex.plymouth.edu/
>Plymouth, NH 03264
>
---------------------------------------------------------------------------
Tim Doggett
Assistant Professor
Texas Tech University
Atmospheric Science Group, Department of Geosciences
Phone: (806)742-3477
Fax: (806)742-1738
---------------------------------------------------------------------------