[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20011223: realtime FTPs to motherlode.ucar.edu (cont.)



>From: "Kevin Polston" <address@hidden>
>Organization: NWSTC
>Keywords: 200111302138.fAULcON07166 motherlode IDD FTP

Kevin,

>I am unsure of how to use LDM or set it up and have been searching for
>some help (notably a friend of mine who works at AWC. He says he knows
>just a little bit about LDM). So I have been waiting to respond until I
>could find out something a little more definite. Obviously I would want
>to get it set up to continue to receive data but since I am not a
>computer programmer I would likely need help.

We can help you with your LDM installation/configuration.

>I did cut way back on some
>things I was downloading and thought that might help out your situation
>on motherlode. It sounds like it didn't really make much difference. 

No, the number of log entries being created is still a problem.

>So why don't you send me some information on what I need to get started
>setting this up.

Please read through our online LDM pages at:

http://www.unidata.ucar.edu/packages/ldm/index.html

>Hopefully we can work through this with a minimal
>amount of problems and then everyone will be happy. 

Right.

>One other thing....I don't understand why the FTP method is less
>efficient than the LDM. What makes it different and why would it be more
>effecient? 

What you have setup is a system that is doing FTPs literally every
second, most of which don't find anything new and simply exit.  This
kind of "polling" is extremely inefficient.  It is the same kind of
thing that has severely hampered the service available on NCEP/OSO FTP
servers.

The LDM, on the other hand, pushes data to a downstream site only when
there is new data.  There is no continuous interrogation of whether or
not there is new data.  Here is an overly simplistic way of stating the
contrast between the two techniques:

Suppose you have a store of data that is being updated from an external
source whenever that data is available.  Which would be a more
efficient use of your time;

o someone asking you once-per-second (or more) if you have any new data
o you sending data to the person when you know you have new data

The high temporal frequency FTPs are essentially performing the first
action: asking once-per-second if there is anything new.  The LDM acts
as the second; it sends data when there is new data to send.  The LDM
way of distributing data uses less CPU on both machines and less
network bandwidth (since things are sent only when needed).

>Have a good holiday and I look forward to hearing from you again.

We can work on getting you setup with an LDM the middle of next week.

What you shold do in the mean time is:

1) identify a machine to run the LDM on; this should be the same one that
   you are currently FTPing data to.  This will need to be a Unix machine.

2) verify that the machine has properly configured DNS service: you must
   be able to do forward and reverse name lookups for the machine:

nslookup full_host_name

should result in the same infomation as:

nslookup host_IP_address

   Without this, the LDM will not work.

3) if your machine is behind a firewall, you must get your network people
   to open up traffic for port 388.  This is not an option.

4) The process of loading the LDM software takes very little time as long as
   you are running a machine for which we have a pre-built binary.  If you
   do not, then you will need a development environment to build the LDM
   binaries from source.

5) a couple of the installation/configuration steps for the LDM requires
   'root' privilege.  If you do not have this for your machine, you will
   need to get your system administrator to complete those steps.

6) as I noted above, you should read through the LDM web pages in order
   to become more familiar with what the LDM is; how it works; and
   why it is useful.

One last question mostly for my own interest:  Since you are an NWS
site, it would seem like you would have a NOAAPORT system available to
you locally.  If this is the case, we don't understand why you would be
FTPing the GINI imagery data available in NOAAPORT channels 1 & 2 from
us.  Do you _not_ have a NOAAPORT receive system available to you at
the NWS Kansas City Training Center?  If not, it would seem that you
are not being provided the tools necessary to do your job!

Tom Yoksas