Comments on the IDD from the Sites Using It
This is simply a collection of email messages from the sites
involved in the IDD testing and deployment. The goal was to
determine how well the IDD was serving the user community
from the point of view of the users themselves. Please bear
with the rudimentary formatting of this collection and with
Ben's poor spelling in the original request for input.
Request for Input from Ben Domenico
Date: Mon, 12 Dec 1994 09:50:58 -0700 (MST)
From: Ben Domenico
To: ldm-users@unidata.ucar.edu
Subject: How goes the IDD?
Hi,
We've been at this IDD deployment into the community
for about 6 weeks now. From my perspective, I can
keep track of things my monitoring the statistics
reports and information about new sites coming online.
However, I'm very interested in your impressions of
the system from he perspective of the people who
are actually running the computers and using the
data (or at least serving the people at your site
who use the data).
How is it going in general?
Is the reliability acceptible?
How much of a burden is it on the staff to keep the system
running?
How much of a burden is it on your network?
How about load on your computer systems?
Are you having any specific problems we should be aware of?
Do you have suggestions for improving the system?
Any other comments that would help improve the IDD
would be welcome.
Of course, if the system is working well for you, we'd
be happy to hear that too.
Thanks again for all your efforts.
--Ben
******************************************************
Ben Domenico Unidata Program Manager
http://my.unidata.ucar.edu P.O. Box 3000
ben@unidata.ucar.edu Boulder, CO 80307
(303)497-8631 FAX: (303)497-8690
David Wilensky from LSU
From: "David E. Wilensky"
Date: Mon, 12 Dec 1994 11:15:48 -0600
Subject: Re: How goes the IDD?
On Dec 12, 9:50am, Ben Domenico wrote:
> Subject: How goes the IDD?
Hi Ben - Mitch called me on Thursday of last week - we're working on some
statistics gathering. You're pretty familiar with our site but I'll fill in
below for the record.
> Hi,
>
> We've been at this IDD deployment into the community
> for about 6 weeks now. From my perspective, I can
> keep track of things my monitoring the statistics
> reports and information about new sites coming online.
> However, I'm very interested in your impressions of
> the system from he perspective of the people who
> are actually running the computers and using the
> data (or at least serving the people at your site
> who use the data).
>
> How is it going in general?
Good - currently using the alternate feed site.
>
> Is the reliability acceptible?
Yes at this level
>
> How much of a burden is it on the staff to keep the system
> running?
Almost none.
>
> How much of a burden is it on your network?
Almost none.
>
> How about load on your computer systems?
< 5% of the loading is IDD specific
>
> Are you having any specific problems we should be aware of?
Already being worked on - thanks.
>
> Do you have suggestions for improving the system?
Not yet
>
> Any other comments that would help improve the IDD
> would be welcome.
>
> Of course, if the system is working well for you, we'd
> be happy to hear that too.
Over all it is great!, I actually did tail -f on both our Ku band feed and the
IDD and watched the IDD arrive timely and in good shape. Thanks for all the
stuff you're doing to pull this off up there.
d. wilensky
>
> Thanks again for all your efforts.
> --Ben
> ******************************************************
> Ben Domenico Unidata Program Manager
> http://my.unidata.ucar.edu P.O. Box 3000
> ben@unidata.ucar.edu Boulder, CO 80307
> (303)497-8631 FAX: (303)497-8690
>
>-- End of excerpt from Ben Domenico
Russ DeSouza from Millersville University
Date: Mon, 12 Dec 1994 12:21:19 -0500 (EST)
From: Russ De Souza
Subject: Re: How goes the IDD?
To: Ben Domenico
Ben
We, at Millersville, seem to be pleased. No general downtime, install
was relatively painless, we can now get HRS data (a HUGE plus), minor
problems by my forgetting to invoke 'ldmadmin reconfigure' upon making
changes. Otherwise, no complaints. Seems reliable and no major drain on
system performanace, outwardly.
Russ De Souza
==============================================================================
Department of Earth Sciences Phone Number: 717-872-3289
Millersville University of Penna. FAX number: 717-872-3985
Millersville, PA 17551-0302 E-mail: rdesouza@snowball.millersv.edu
gopher: snowball.millersv.edu WWW: http://snowball.millersv.edu:8080
==============================================================================
Harry Edmon from the University of Washington
From: Harry Edmon
Subject: Re: How goes the IDD?
To: ben@unidata.ucar.edu (Ben Domenico)
Date: Mon, 12 Dec 1994 09:30:01 -0800 (PST)
>
> Hi,
>
> We've been at this IDD deployment into the community
> for about 6 weeks now. From my perspective, I can
> keep track of things my monitoring the statistics
> reports and information about new sites coming online.
> However, I'm very interested in your impressions of
> the system from he perspective of the people who
> are actually running the computers and using the
> data (or at least serving the people at your site
> who use the data).
>
> How is it going in general?
Good.
>
> Is the reliability acceptible?
Accept for my occasional problem with hung processes.
>
> How much of a burden is it on the staff to keep the system
> running?
Small.
>
> How much of a burden is it on your network?
Seems to be small enough.
>
> How about load on your computer systems?
Acceptable.
>
> Are you having any specific problems we should be aware of?
My occasional problem with hung processes.
>
> Do you have suggestions for improving the system?
None that come to mind at the moment. I am looking forward to the
updated gribtonc.
>
> Any other comments that would help improve the IDD
> would be welcome.
>
> Of course, if the system is working well for you, we'd
> be happy to hear that too.
>
> Thanks again for all your efforts.
> --Ben
> ******************************************************
> Ben Domenico Unidata Program Manager
> http://my.unidata.ucar.edu P.O. Box 3000
> ben@unidata.ucar.edu Boulder, CO 80307
> (303)497-8631 FAX: (303)497-8690
>
>
--
Harry Edmon INTERNET: harry@atmos.washington.edu
(206) 543-0547 FAX: (206) 543-0308
Dept of Atmospheric Sciences, AK-40
University of Washington
Angel Li from Miami
Date: Mon, 12 Dec 1994 12:38:18 -0500 (EST)
From: Angel Li
Subject: Re: How goes the IDD?
> How is it going in general?
>
It's going really well and that's especially great for me because I
tend not to babysit the LDM much. I usually wait for someone to point
out to me that soemthing's not working e.g. "Hey, where's the forecast?"
> Is the reliability acceptible?
I'm very happy with the reliability of the current LDM since we switched
to running it on an Alpha.
> How much of a burden is it on the staff to keep the system
> running?
>
Barely none for me as long as the disk doesn't get filled by someone else.
> How much of a burden is it on your network?
My guess is that the IDD doesn't load the network much.
> How about load on your computer systems?
On the Alpha that's hosting the LDM it's just another two or three background
processes. I haven't heard Otis complain (it's his workstation).
> Are you having any specific problems we should be aware of?
>
The McIdas ingesters need to be ported to the Alpha. I'm willing to attempt
it when I get some free time. Tom mentioned it wasn't going to be pretty.
Angel
David Knight from SUNY Albany
Date: Mon, 12 Dec 1994 17:56:26 GMT
From: knight@atmos.albany.edu (David J. Knight)
Subject: Re: How goes the IDD?
Ben,
> Hi,
>
> We've been at this IDD deployment into the community
> for about 6 weeks now. From my perspective, I can
> keep track of things my monitoring the statistics
> reports and information about new sites coming online.
> However, I'm very interested in your impressions of
> the system from he perspective of the people who
> are actually running the computers and using the
> data (or at least serving the people at your site
> who use the data).
>
> How is it going in general?
Once we got a realiable feed, in general everything is going well.
>
> Is the reliability acceptible?
Yes. The only problem is that delays receiving products can
be long. We occasionally have periods where data is delayed
by 20 min or more. Seems to happen most often on fridays,
and when there is a storm in the area.
>
> How much of a burden is it on the staff to keep the system
> running?
not too bad
>
> How much of a burden is it on your network?
>
> How about load on your computer systems?
neither of the above seems to be significant
>
> Are you having any specific problems we should be aware of?
Just the long latentcy time mentioned above
>
> Do you have suggestions for improving the system?
note really
>
> Any other comments that would help improve the IDD
> would be welcome.
overall I would say this has been a big success
>
> Of course, if the system is working well for you, we'd
> be happy to hear that too.
>
> Thanks again for all your efforts.
And thanks for yours
> --Ben
> ******************************************************
> Ben Domenico Unidata Program Manager
> http://my.unidata.ucar.edu P.O. Box 3000
> ben@unidata.ucar.edu Boulder, CO 80307
> (303)497-8631 FAX: (303)497-8690
David
David Knight Tel: (518)-442-4204
Department of Atmospheric Science Fax: (518)-442-4494
SUNYA ES-228 Bitnet: dk962@albnyatm.bitnet
Albany, NY 12222 Internet: knight@atmos.albany.edu
>
Tom Kitterman from Florida State
Date: Mon, 12 Dec 1994 13:00:21 +0500
From: kitt@maxwell.met.fsu.edu (Tom Kitterman)
Message-Id: <9412121800.AA02923@maxwell.met.fsu.edu>
To: ben@unidata.ucar.edu
Subject: Re: How goes the IDD?
> From @huey.met.fsu.edu:ben@unidata.ucar.edu Mon Dec 12 12:09 EST 1994
> Date: Mon, 12 Dec 1994 09:50:58 -0700 (MST)
> From: Ben Domenico
> To: ldm-users@unidata.ucar.edu
> Subject: How goes the IDD?
> Mime-Version: 1.0
>
> Hi,
>
> We've been at this IDD deployment into the community
> for about 6 weeks now. From my perspective, I can
> keep track of things my monitoring the statistics
> reports and information about new sites coming online.
> However, I'm very interested in your impressions of
> the system from he perspective of the people who
> are actually running the computers and using the
> data (or at least serving the people at your site
> who use the data).
>
> How is it going in general?
>
I would say just ok.
> Is the reliability acceptible?
>
Marginally acceptable. Our feed is from wx3 (used to be wx2).
John has been having some problems I know. I also have
been having some problems upgrading my server to Solaris 2.3,
so the guys downstream from me might also complain abit.
I've finally got the machine running 2.3 last thursday so my
ldm should be stable now. My problems with the machine had
nothing to do with the ldm, just problems getting 2.3 to work
with the network routing/nfs server software.
There have been a couple of times when my queue file got
corrupted and data stopped flowing. Some loss of data
occurred then.
> How much of a burden is it on the staff to keep the system
> running?
>
No more of a burden that the old system was. In some ways it
is easier to check out and change stuff, restart, etc.
> How much of a burden is it on your network?
>
I have been told by our campus network administrator that our
IDD server has 'noticeably' more traffic than their campus
www/news server. I have not seen any major decrease in network
response though.
> How about load on your computer systems?
>
No noticable increase.
> Are you having any specific problems we should be aware of?
>
None other than mentioned above.
> Do you have suggestions for improving the system?
>
Once, I think the system stopped and left the ldm
registered with the rpc port mapper, requiring the
use of the rpcinfo program to delete it from the map.
This program requires root priveledges. It would be
nice to fix things so the ldm user could do this,
perhaps a setuid version of rpcinfo?
Tom Kitterman internet: kitterman@met.fsu.edu
Meteorology Department phonenet: (904) 644-1550
Florida State University sneakernet: 408 Love Building
Tallahassee, Fl. 32306-3034 faxnet: (904) 644-9642
Eric Haug from Saint Louis University
Date: Mon, 12 Dec 94 13:10:51 CST
From: ejh@slueas.slu.edu (Eric J. Haug)
Subject: idd response
> From: Ben Domenico
> Subject: How goes the IDD?
> How is it going in general?
Fine.
>
> Is the reliability acceptible?
We have not had any software problems.
Outages at our feed site have caused 99% of the data loss.
A pulled wire locally caused the other outage.
>
> How much of a burden is it on the staff to keep the system
> running?
not much.
>
> How much of a burden is it on your network?
Unknown
>
> How about load on your computer systems?
A machine is sort of dedicated to this task.
>
> Are you having any specific problems we should be aware of?
Seeing our site name in the routing information
We currently have a sparc1 with 40 Meg memory running the system.
I have noticed that running the ldmadmin ensure every 10 minutes
seems to cause more problems than it is worth. I think that this
causes the system to page too much. I just did not have the time
to check it out, so i turned it off. We intend to run the
system on a sparc 5 with 48 Meg memory soon.
>
> Do you have suggestions for improving the system?
Most of my immediate desires are scheduled in later versions.
Though if we could search for alternate feeds during outages.....
>
> Any other comments that would help improve the IDD
> would be welcome.
We need to remember to communicate with upstream/downstream sites.
>
Thanks again for all your efforts.
Eric
Dean Vickers from Oregon State
From: vickers@ATS.ORST.EDU (Dean Vickers)
Subject: Re: How goes the IDD?
Date: Mon, 12 Dec 94 11:20:49 PST
According to Ben Domenico:
>
> How is it going in general?
good, we are pleased so far
> Is the reliability acceptible?
yes, with our poor situation before, the reliability is better.
our sat receiver and/or reception was never good enough to get
more than 3 or 4 full sat images a day, now we get them all.
> How much of a burden is it on the staff to keep the system
> running?
very small
> How much of a burden is it on your network?
it is alot of traffic, but network is holding up quite well
> How about load on your computer systems?
practically nothing on a sparc5 with 32mb memory and 4.1.3
> Are you having any specific problems we should be aware of?
> Do you have suggestions for improving the system?
it would be nice if the wisconsin channel sat image decoders (lwtoa3)
were included in the package
> Any other comments that would help improve the IDD
> would be welcome.
>
> Of course, if the system is working well for you, we'd
> be happy to hear that too.
working nicely
-Dean
--
Earle Ripley from the University of Saskatchewan
Date: Mon, 12 Dec 1994 13:15:27 +0000 (GMT)
From: eripley@marathon.usask.ca (Earle Ripley)
Subject: Re: How goes the IDD?
To: ben@unidata.ucar.edu
Here I my reaction to the IDD--
>
> Hi,
>
> We've been at this IDD deployment into the community
> for about 6 weeks now. From my perspective, I can
> keep track of things my monitoring the statistics
> reports and information about new sites coming online.
> However, I'm very interested in your impressions of
> the system from he perspective of the people who
> are actually running the computers and using the
> data (or at least serving the people at your site
> who use the data).
>
> How is it going in general?
>
In general, I am pleased with it.
> Is the reliability acceptible?
>
It seems to be just as reliable as the C-band satellite feed
> How much of a burden is it on the staff to keep the system
> running?
>
The only staff I have is me, and I am holding up so far.
> How much of a burden is it on your network?
>
None, that I know of.
> How about load on your computer systems?
>
No complaints yet!
> Are you having any specific problems we should be aware of?
>
There seem to be a lot of gaps in the McIdas data.
> Do you have suggestions for improving the system?
>
No!
> Any other comments that would help improve the IDD
> would be welcome.
>
Although I have only the flashing lights to go by, the C-band feed seems to be still
operational; is that right? I bought a ku-=
> Of course, if the system is working well for you, we'd
> be happy to hear that too.
>
I'm happy!
Earle Ripley>
Note from Steve Emmerson at UPC
From steve@unidata.ucar.edu Mon Dec 12 11:13:10 1994
Full-Name: Steve Emmerson
Subject: Re: How goes the IDD? (fwd)
Date: Mon, 12 Dec 94 11:13:07 -0700
From: steve@unidata.ucar.edu
In the above message you wrote:
>> Do you have suggestions for improving the system?
>>
>Once, I think the system stopped and left the ldm
>registered with the rpc port mapper, requiring the
>use of the rpcinfo program to delete it from the map.
>This program requires root priveledges. It would be
>nice to fix things so the ldm user could do this,
>perhaps a setuid version of rpcinfo?
Already in the next release.
--------
Steve Emmerson
John Kemp at the University of Illinois
Date: Mon, 12 Dec 1994 14:30:48 -0600
From: JohnKemp@uiuc.edu (John Kemp)
Subject: Re: How goes the IDD?
>How is it going in general?
>
>Is the reliability acceptible?
>
>Do you have suggestions for improving the system?
>
It's going real well. I just thought I would raise
the issue of secondary sites again. Since UNIDATA controls
the topology, it would be easiest for UNIDATA to specify
secondary sites for everyone. A policy could then be put in
place which would dictate when it is recommended or not recommended
to switch to the secondary site. Switching can be done manually.
A case in point...
Univ of Illinois had a nasty hardware problem show up on
our primary data machine for IDD, wx2.atmos.uiuc.edu. That
caused us to move our operation to another machine. The
process of moving gave us a few hiccups, most of which I
think are now cured. But in the process of discovering
the hardware problem, and then moving the entire setup to
another workstation, we have had a number of 6-12 hour outages,
usually in the night.
Machine failures happen. Network failures happen. When these
things happen for extended periods the IDD is impacted. Product
queing is fine, but for extended outages is does nothing.
We need to have secondary sites that we can switch
to when upstream sites go down for extended periods.
Again, I think the IDD is a huge success. I just think we have
this significant point-of-failure that needs to be plugged at
some point.
2cents,
JohnKemp@uiuc.edu
-------- john kemp ( ( )_ internet - johnkemp@uiuc.edu
----- ( ( __)
--- univ of illinois (_ ( __)
-- dept of atmos sci .(____).
- 105 s gregory ave ... phone - (217) 333-8390
- urbana, il 61801 ... fax - (217) 244-4393
Dan Vietor at Purdue
Date: Mon, 12 Dec 1994 15:51:20 -0500 (EST)
From: Daniel Vietor
Subject: Re: How goes the IDD?
On Mon, 12 Dec 1994, Ben Domenico wrote:
> How is it going in general?
Purdue University
Running LDM 4.1.36 on IBM RS/6000 model 320 under AIX 3.2.0
We are fighting some general network problems here. The LDM software has
been running well until we turned into a secondary relay site. The
additional load has caused broken (our AIX based LDM server) to crash on
several occassions. I am trying to trace down the problem. Here is one
thing I have done to reduce the problem. First, check the max transmits
queued with:
netstat -v
Hardware Address: 02:60:8c:2d:3e:64
Transmit Byte Count: 248409348.0 Receive Byte Count: 3312357989.0
Transmit Frame Count: 4912529.0 Receive Frame Count: 6654179.0
Transmit Error Count: 0 Receive Error Count: 0
Max Netid's in use: 7 Max Transmits queued: 34
...
Check the Max Transmits value. Now you need to check the queue buffer
size to see if it may be overrunning. Run:
lsattr -E -l ent0
This lists the effective values for the ethernet driver on AIX. Check
the line:
xmt_que_size 90 TRANSMIT queue size
This should be at least 90 to ensure the buffer is big enough to cache
all outgoing packets. The default for 3.2 is 30 and 3.2.5 is 64. To
increase queue size, run:
chdev -l ent0 -a xmt_que_size=90 -P
This changes the online database so that at the next reboot, the changes
will take effect. This should make sure you have enough buffer to keep
the system from crashing routinely. It doesn't solve the problem, it
just reduces it.
> Is the reliability acceptible?
Overall, the system has been reliable when the operating system hasn't
kernel panic'd on the network load. Again, this is something I am
working on.
> How much of a burden is it on your network?
So far it hasn't been a significant load on the network but could be
combined with gopher, WWW and ftp traffic.
=============================================================================
Daniel Vietor INTERNET devo@cell.atms.purdue.edu
Dept of Earth & Atmospheric Sciences TITLE Senior Project Specialist
Purdue University WXP Developer
West Lafayette IN 47907 GOPHER thunder.atms.purdue.edu
PH 317-494-3292 FAX 317-496-1210 WWW http://thunder.atms.purdue.edu
=============================================================================
Steve Chiswell at University of Hawaii and North Carolina State
Date: Mon, 12 Dec 1994 12:41:43 -1000
From: Steven R. Chiswell
Subject: Re: How goes the IDD?
In general:
At Hawaii the IDD has been fine under SunOS5.3. Our network from
the mainland has gone down a few times. This is a continuing problem
in general, not with the IDD- except that we really only have
one path to our upstream site, so a networkoutage cannot be circumvented.
At ncsu:
Now that dmost downstream sites have discontinued feeding, the IDD is stable.
The major problem appears to be system load. In particular, the LWTOA3
action is a major drain on the system. On a few occasions, I have found
10-15 instances of LWTOA3 running on the system at the same time.
This seems to be after the SSEC has been down, and returns after an outage.
We appear to get peppered with back-missed products. This has a downward
spriraling effect, so that the system is so backlogged, all actions
including gribtonc, gribtogem etc get very behind. If nothing
is done, the system runs out of swapspace, or no more processes.
upstream:
I have seen messages from alden when one of the ingestors
has crashed. I have seen PPS,IDS and HRS all presumably
go down.
of general concern:
I discussed this with Mitch recently. For those of you
who may not have noticed, many of PHNL's products are
impropperly transmitted so that the line:line:ctrl-c:ctrl-a:nextproduct
sequence is violated.
The effect of this is that whatever product follows a Honolulu
product (generally forecasts) gets processed as part of the
Hawaii product (and missed as a separate product). If you
miss a critical tornado warning etc, try looking wherever
you are stashing hawaii forecasts (if you save them at all).
Note: the product you will be missing is not the hawaii product,
but the product that follows is considered part of the hawaii product.
My solution for this is to create an action which passes hawaii
products (PHNL) to a short c program. If a second product is
found in the message (Ie a second SOH-ETX pair), the product
is spit out to a file, which is then reingested locally using
pqing.
Other notes:
Thanks for the quick repair on the upstream injection of RUC
products (with ammended headers).
Steve Chiswell
chiz@lumahai.soest.hawaii.edu
Jim Mintha at the University of British Columbia
From: Jim Mintha
Subject: How goes the IDD
To: ben@unidata.ucar.edu
Date: Mon, 12 Dec 94 23:58:23 PST
Hi Ben,
> We've been at this IDD deployment into the community
> for about 6 weeks now. From my perspective, I can
> keep track of things my monitoring the statistics
> reports and information about new sites coming online.
> However, I'm very interested in your impressions of
> the system from he perspective of the people who
> are actually running the computers and using the
> data (or at least serving the people at your site
> who use the data).
>
> How is it going in general?
> Is the reliability acceptible?
> How much of a burden is it on the staff to keep the system
> running?
So far the system has been running great, it seems very reliable, and
requires next to no maintenance (although we are just a leaf at the
moment) Installation was quite easy, far easier than most other
packages/systems I have installed, and its fairly easy to use.
> How much of a burden is it on your network?
> How about load on your computer systems?
The network load is quite small for us (again data is only flow TO
us), and load on our Sparc 5 is negligible
> Are you having any specific problems we should be aware of?
> Do you have suggestions for improving the system?
No problems thus far, the only suggestions that I can offer are
additions to the ldm admin guide (both are fairly minor):
1) A (larger) note about tabs in the config files. I just pasted
config lines from the email I got from you guys, unfortunately tabs
got turned into spaces along the way.
2) A mention of the message: "Couldn't get arguments for RPC request"
is not extremely serious, and just indicates network problems between
you and your upstream site.
Overall you guys have done a tremendous job. In an industry where
deadlines are almost never met, you had just about everyone up on
Dec. 1
Jim
Jim Mintha (mintha@geog.ubc.ca) Home: (604) 731-7240 or 737-6094
Geography System Administrator Work: (604) 822-2174 or 465-5074
Fax: (604) 822-6150 or 465-6799
>> SARCAST \'sar-kast\ v. 1. To engage in the art of sarcasm <<
Chi-Pai Chou and Michigan State
Date: Tue, 13 Dec 94 09:30:21 EST
From: Chi-Pai Chou
To: ben@unidata.ucar.edu
Subject: Re: How goes the IDD?
Michigan State University
Running LDM 4.1.36 on Sun SPARCstation IPX under SunOS 4.1.3
We are pretty satisified with the LDM 4.1.36.
Installation was relatively painless.
No general downtime. No major drain on system performance.
IDD seems reliable.
_______________________________________________________________________________
Chi-Pai Chou Department of Geography
Site Manager Michigan State University
chouchi@nimbus.geo.msu.edu
Rodger Getz at Auburn
Date: Tue, 13 Dec 1994 09:30:11 +0600
From: rgetz@awis.auburn.edu (Rodger R. Getz)
Subject: Re: How goes the IDD?
>How is it going in general?
We are pleased with the conversion to the IDD. We are currently
depending on the Ku service to maintain our operational databases and
are running the IDD system on a separate filesystem. This has enabled
us to slowly transition operations. We have a rather robust pqact.conf
that fires up model runs, forwards a number of products to other processes,
runs locally developed programs, etc. These tasks are being switched over
a little bit at a time. This has required some minor recoding of scripts
and programs.
>Is the reliability acceptible?
There were some problems during the first week due to hardware failures
upstream. Since then, the reliability has been been excellent. Our
upstream site (FSU) has done a good job in letting us know when they would
be down.
>How much of a burden is it on the staff to keep the system
>running?
What staff? Just kidding. We followed all the instructions in setting up
the IDD and slowly added tasks to gain confidence. The burden has been
minimal.
>How much of a burden is it on your network?
We have not noticed any burden on our network. Our campus network
administrator has monitored us and has confirmed no impact on the
campus backbone.
>How about load on your computer systems?
The IDD is running on my workstation (SPARCstation 20) which I operationally
use every day (and night). If there was a performance problem, I would be
the first to complain!
>Are you having any specific problems we should be aware of?
No. We still have a few machines running SunOS, so trying to maintain the
old OS along with moving applications to run under Solaris has been a
pain at times.
>Do you have suggestions for improving the system?
No.
>Any other comments that would help improve the IDD
>would be welcome.
No.
>Of course, if the system is working well for you, we'd
>be happy to hear that too.
You guys and gals have done a remarkable job! We are looking forward to
making use of the HRS products, something that we could not afford under
the old system. The ldmadmin interface has been a great feature, making
it simple to work with the ldm. We plan to interface some local data
feeds into the ldm which will simply our management and make for a
consistant interface. The IDD is one of the best examples of the
"Unidata Community" where we are all helping each other do a better job
with fewer resources.
Season's greetings to everyone at Unidata!
---------------------------------------------------------------
Rodger R. Getz Internet: rgetz@awis.auburn.edu
Meteorologist-in-Charge
NOAA/National Weather Service (205) 844-4514 (voice)
SE Ag Weather Service Center (205) 887-4511 (FTS)
Auburn University, Alabama (205) 844-5933 (FAX)
Snail-Mail: National Weather Service, PO Box 3267, Auburn, AL 36831-3267
www: http://www.awis.auburn.edu gopher: gopher.awis.auburn.edu
Mike Leuthold at Arizona
Date: Tue, 13 Dec 1994 8:56:42 -0700 (MST)
From: LEUTHOLD@ATMO1.ATMO.ARIZONA.EDU (Mike Leuthold, University of Arizona)
Subject: RE: How goes the IDD?
To: ben@unidata.ucar.edu
Ben...
>How is it going in general?
Very well. The transition from 4.0 on Ultrix to 4.1 on AIX was smooth.
>How much of a burden is it on the staff to keep the system
>running?
Just a little during configuration and startup, but that is to be expected.
Other than that, minimal. In general, 4.1 seems easier to use and configure,
mainly because of ldmadmin.
>How much of a burden is it on your network?
I don't see any additional load.
>How about load on your computer systems?
Small, An occasional gribtogem or lwtoa3 shows up for a few moments but it is
nothing that impacts system performance.
>Are you having any specific problems we should be aware of?
None.
>Do you have suggestions for improving the system?
Not at this time.
>Of course, if the system is working well for you, we'd
>be happy to hear that too.
It seems very stable and reliable (according to the stats). Thanks for all the
hard work on IDD from everyone at UPC.
Mike
Rob Cermak at Rutgers
Date: Tue, 13 Dec 94 11:21:57 EST
From: Rob Cermak
To: ben@unidata.ucar.edu
Subject: Opted not to put to ldm-users
Ben,
Here's my reply to your mail...I opted not to put it to ldm-users.
----
>How is it going in general?
In general, quite well. I think the network jams have been found and
corrected. The deployment of IDD has delayed other projects such as
scratching my head on why McIdas-X works to a certain point on a Linux
box...and/or by the BSD boot disks plainly reset the Gateway 2000 computers.
It was well worth the effort to convert to IDD!
Sometime Summer '95, the department will be moving buildings...
as the time grows nearer more information will be posted under our
IDD status page (http://vortmax.rutgers.edu/idd/)
>
>Is the reliability acceptable?
Quite. I'm sure the reliability is about the same or better than the
actual satellite reception. It will be a bit longer to transition from
satellite to IDD, but bit quicker due to the less than a year lifetime of
McIdas data transmission via satellite. RUCS runs a 'wxmap' server that
currently seems to be holding its own on the KU-DDS at the higher speed.
>
>How much of a burden is it on the staff to keep the system
>running?
Currently, its a one person effort on 15/hours a week...though some of
the mail sent to support may indicate that I've worked more than that
trying to keep up with the tide.
There is no telling what sort of support to expect from Rutgers until
the move happens. It's generally bad news when two departments merge.
Rest assured, if I have anything to say about it, that downstream sites
will not have a data disruption of any significant magnitude. I
think all our sites failover to FSU, which is a bonus.
>
>How much of a burden is it on your network?
No grumbles yet from our network support group. I'm sure if it were
a problem, they would have let us know by now.
>
>How about load on your computer systems?
Load is minor except for HIYA errors and/or Refusal of connections.
HIYA errors have a sleep mechanism built into them, but if a site
continually asks for a connection, it is refused and not but a minute
later another request is generated. Could there be a small delay
build between requests from the downstream node?
>
>Are you having any specific problems we should be aware of?
The ldmadmin program generates a decent amount of load on the system.
So far the running the ldmadmin ensure crontab has not failed except for once.
The only time this happened, the ldmadmin jobs stacked up and pinned the
CPU to 100% and load up to 2.00.
This is minor to having 100 users on a Sparc10 with loads up & above 10.
Wiping out the ldmadmin jobs and reading the nice e-mail was the eventual
solution. Make sure the lock file is gone...
I think you might find that one of the ldmadmin ensure cronjob times
conflicts with one of the other cronjobs (starting at the same minute
after the hour). Two copies of ldmadmin cannot be running. Make sure
there is about 3 to 5 minutes between the minutes in all the ldmadmin
crontab entries. If there is a conflict, it could tie up the program.
***
Any information on how spectra.ceo.yale.edu is behaving/working and/or not
working? Is the problem on our end?
The LDM is continuing to log 0:00 CPU time and collecting HIYA errors.
http://vortmax.rutgers.edu/cgi-bin/ldmstat?spectra
***
>
>Do you have suggestions for improving the system?
No suggestions here. The newer tools have put us behind again
due to usage of Motif. That and the newer package of XFree86
included a new format for the Xconfig files.
The bright side of this is that we will eventually have the right
tools for the right job. Motif and ansi-C as well as Solaris 2.4.
Rutgers has converted some machines over to 2.4, but I have no idea
what patches they have...(These might be beta 2.4 /w patches).
I've also been notified that a CDROM version of 2.4 (non-beta) is on order.
Needless to say, I'm quite ready to backup the LDM portion of our disk
and do a full re-install over 2.3...
Happy Holidays!
Rob Cermak
Rutgers Univ.
Steve McGee at NMSU
Date: Tue, 13 Dec 94 12:24:39 MST
From: smcgee@akbar.psl.nmsu.edu (Steve Mcgee)
Subject: ldm
In general I am quite pleased with LDM. It is reliable and quiet.
One question I have is - ughhhh. - OS/2. Is there an LDM that I
can feed my local OS/2 machine with? Right now I have a local
ftp script which I invoke upon receipt in the pqact config.
This ports files to the OS/2 machine which executes a McBasi script
every five minutes to check for new data. It seems to work ok but
it's a little crude.
Kudos for ldm. It's AOK.
Steve McGee smcgee@psl.nmsu.edu
Robert White at Texas A&M
From: rwhite@anvil.tamu.edu (Robert White)
Subject: Re: How goes the IDD?
To: ldm-users@unidata.ucar.edu
Date: Tue, 13 Dec 1994 16:07:46 -0600 (CST)
How is it going in general?
Here at Texas A&M, the ldm and idd setup is running on an SGI Indigo2
running IRIX 5.2. It has been extremely stable and reliable since
I have set it up. This server is a lab server for 15 SGI Indy SC's
and does NFS, NIS, and DNS service to these and 3 other dept. SGI's
as well.
How much of a burden is it on the staff to keep the system
running?
Very little. I hardly look at it anymore. (hint: I'm about the only staff here)
Thanks a bunch for this!
How much of a burden is it on your network?
Not much. Most of the burden seems to be on the disk I/O when decoding
for gempak and surf_split.
How about load on your computer systems?
I have noticed a degradation of performance on the server when the lab
machines we have tied into it are being used as well to look at the
data, but performance is acceptable.
Are you having any specific problems we should be aware of?
None that I'm aware of, but I'll let you know if I do.
Of course, if the system is working well for you, we'd
be happy to hear that too.
We're happy.
--
###############################################################################
Robert White #
Dept. of Meteorology # Seek the Lord while He may be found.
Texas A&M University # Call upon Him while He is near.
College Station, TX # Let the wicked forsake his way, and
77843-3150 # the unrighteous man his thought.
phone: (409) 845-7646 #
fax: (409) 862-4466 # Isaiah 55:6-7
e-mail: rwhite@coriolis.tamu.edu
###############################################################################
Donna Tucker at Kansas
Date: Tue, 13 Dec 1994 17:37:10 +0600
From: ldm@chinook.phsx.ukans.edu
To: ben@unidata.ucar.edu
Subject: Re: How goes the IDD?
How is it going in general?
Well we are just getting started here but I guess it is going OK. I spend
a lot of time tracing things that seem to be going wrong to determine why.
Is the reliability acceptible?
Yes, it helps to be able to switch the feed to another site if the
primary feed is down.
How much of a burden is it on the staff to keep the system
running?
It is hard to tell at this point. We are still trying to get everything
configured as we would like and that's a big job. We will see how it is
when things settle down.
How much of a burden is it on your network?
They tell me it is no problem. We have had network problems here for the
past week and they are still tracking down the source but they have assured
me that the LDM computer is not contributing to the problem (Whew!).
How about load on your computer systems?
Can't notice it is running without checking
Are you having any specific problems we should be aware of?
Nothing new I can't handle right now but I'm sure there will be more to come.
Do you have suggestions for improving the system?
FAQ list, examples of how other universities have configured the data files
directory.
Please convey thanks to everyone who helped get our system running.
Please let me know if there is any other information I can provide.
Donna Tucker
ldm@chinook.phsx.ukans.edu
Larry "The Weather Man" Riddle at UCSD/Scripps
Full-Name: General Support
To: mitch@unidata.ucar.edu, ben@unidata.ucar.edu
Subject: 941214:LDM Questionnaire
Reply-To: weather@weather.ucsd.edu
Date: Wed, 14 Dec 1994 10:16:23 -0700
From: Unidata Support
------- Forwarded Message
Subject: 19941214
LDM Questionnaire
>From: weather@weather.UCSD.EDU (Larry "The Weather Man" Riddle)
>Organization: .
>Keywords: 199412141656.AA06323
How is it going in general?
Poorly. We saw "first light" last wednesday (Dec 7th, the
date may be significant). Since that time, we've been able
to keep it on-line only about 20-25% of the time.
I don't know if this is a problem with the UPC's DEC Alpha
binaries, DEC Alphas in general, or the way we're trying
to run it. I just know it's driving us crazy.
Is the reliability acceptible?
No. I just don't know what can be done about it.
How much of a burden is it on the staff to keep the system
running?
As with most other sites, there is no "staff" here to do
this, just two of us with "additional duties".
One is the site administrator Bill Higgins (a "one-deep"
position), with responsibility for approximately 28 unix/osf1
cpus, two file servers, two networks (copper and glass), 7+
networked high volume printers, and various other computer
assets (including several PCs and Macs). He doesn't have a
lot of spare time.
The other one is me, a non-tax-supported climatologist who
is getting farther behind in the research that pays for all
of these toys.
We have almost no state or non-overhead university support
here since we have almost no teaching load. We live almost
exclusively on contracts and grants, after the university
takes over half for "overhead". Salaries for direct staff
are grant supported, not overhead.
How much of a burden is it on your network?
Insignificant, as far as I can see, when it's up. But then,
it shouldn't be. We have T3 connectivity with UCLA (although
they don't appear to be using the S2K net to feed us yet) and
glass fiber in-house.
How about load on your computer systems?
The cpu running ldm is a DEC Alpha 3000/800 file server. It's
high on disk space and low on cpu users. Not significant.
Are you having any specific problems we should be aware of?
We're working with Mitch Baltuch on our problems here. It's
probably driving him crazy, too. I had a beer with him, the
last time I was in Boulder. I get the feeling that I may
not be welcome in Boulder, ever again.
Do you have suggestions for improving the system?
Is it possible to build any system that is as powerful as
the ldm has to be (and as flexible) and still make it simple
enough for computer duffers like me to run?
Any other comments that would help improve the IDD
would be welcome.
Since I don't know why it won't run...
Of course, if the system is working well for you, we'd
be happy to hear that too.
I feel that the facts here at UCSD should not overshadow
the facts elsewhere. Most users seem to be having little
or no trouble with the system. Mostly, it works.
I think we should all step back and look at what has been
accomplished. The UPC brought a very dynamic system into
existance almost a year ahead of schedule, with not too
much advance notice of the need for early implementation.
All of this while continuing it's support of several other
major systems (GEMPAK, WXP, etc.). This compares very
favorably with the time frames for most major "turn key"
systems. Thanks, folks.
Larry
---===---=-=-=-=-=-=-=-=-=-=-=====[\/]=====-=-=-=-=-=-=-=-=-=-=---===---
-----===(* Climate's what we expect, but weather's what we get. *)===-----
Larry Riddle : Climate Research Division : Scripps Institution of Oceanography
University of California, San Diego : La Jolla, California 92093-0224
Phone: (619) 534-1869 : Fax: (619) 534-8561 : E-Mail: lriddle@ucsd.edu
------- End of Forwarded Message
Tom Whittaker from SSEC
Date: Wed, 15 Nov 95 20:28:05 CST
From: Tom Whittaker
To: Ben Domenico
Subject: Re: a toast
Ben: My heartiest congratulations to the Unidata staff for
putting this package together and making it fly. That there are
now over 100 sites is a monumental achievment; everyone should
tip a couple over this!
tom
Ben Domenico wrote:
>
> All you IDDers out there:
>
> It's time to take a (brief) break from your labors of
> keeping the data flowing over the less-than-perfect
> Internet. While it is clear that there are still many
> battles left to get more sites online and to
> improve the reliability overall, it's important to
> recognize a milestone that occurred last week when we
> surpassed 100 sites on the IDD. (We're actually
> up to 101 now).
>
> So I'd like to salute all of you who have worked so
> hard to make this happen--especially the relay
> site administrators and the UPC technical staff.
>
> As you might imagine, we're going to schedule a UPC
> "symposium" to celebrate and I hope you all will find
> a proper time and place to join us in spirit(s).
>
> --Ben
> ******************************************************
> Ben Domenico Unidata Program Manager
> http://my.unidata.ucar.edu P.O. Box 3000
> ben@unidata.ucar.edu Boulder, CO 80307
> (303)497-8631 FAX: (303)497-8690
>
> P.S. OK that's enough frivolity for now, so get back to
> pouring over those netcheck logs to figure out
> what's causing our IDD afternoon flakiness.
>
--
Tom Whittaker tomw@ssec.wisc.edu
Space Science and Engineering Center University of Wisconsin-Madison
Phone/VoiceMail: 608/262-2759 Fax: 608/263-6738
ben@unidata.ucar.edu
