[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20031210: Unidata data feed for Vietnam (cont.)

>From: Mai Nguyen <address@hidden>
>Organization: National Center for Hydro-Meteorological Forecasting of Vietnam
>Keywords: 200312020023.hB20N4p2027742 IDD LDM Windows Unix Linux


>We have a problem in our system that the computer with the IP to the 
>Internet is using Windows NT, eventhough other computers that use the 
>data to run our Regional NWP model are on Unix/Linux system.
>However, we are being able to use that WindowsNT computer to get data 
>fed from the German server (DWD) and pull data from Australian server 
>(using ftp) (these sites do supposedly run on Unix/Linux platforms).
>Therefore, is it possible for us to either
>    + pull data from one of your sites, or
>    + for you to push data to our site ?

We believe that our LDM package provides a vastly superior method for
receiving realtime feeds of data.  The IDD project (again, a
distribution network based on cooperating LDMs) has been delivering
data throughout North America (the US, Canada, and Mexico) for about a
decade.  Recently, we improved the LDM's network usage so that we can
deliver realtime data to remote regions like Brazil, Europe, Hong Kong,
and Australia.

>(I believe that the core actions of the scripts to get or push data are 
>based on ftp calls).

The problem with doing FTP pulls is that the FTP scripts typically go
to a site and have to figure out what is new and then transfer the
data.  This may work passingly well for non-realtime data, but it is
very inefficient for data that is changing on a short timeframe (like
realtime feeds of weather data).

The load that the LDM puts on both the receiving and feeding machine(s)
is very small when compared to automated FTP scripts.  This is one
reason that we always advise our users that using FTPs to grab data is
never to be used as a substitute for realtime feeds using the LDM.
This is a standard policy that we have maintained for about as long
as the IDD has been in existence.

>Hope that we can find a good solution for this.

The absolute easiest and, in our opinion, best thing would be for your
site to configure a Unix/Linux box so that it is accessible over the
Internet.  You could then run the LDM and get worldwide data seconds
after it is available.

Another feature of the LDM to consider is its ability to run
processes on data as it arrives.  The module that does this, pqact, is
notified (by signal) that a new product has been received and placed in
the LDM queue.  pqact then looks through a file of pattern/actions for
an action that matches the product's header (using regular
expressions).  If one is found, the action is executed immediately.
Some example actions are:

- filing of products
- mailing of produts to a user or users
- sending the product to a decoder for conversion into a format used
  by a site's display and analysis software
- creation of a display based on the product's content
- etc.

>Thanks you for your help and patience.

No worries.  I am hopeful that you will find the features of internet
data delivered by the LDM to be compelling enough to configure a
Unix/Linux machine to be accessible over the Internet.  If you are only
able to use a Windows machine, we can offer no viable method for
feeding you data at this time.

>Best regards, Mai



>From address@hidden Thu Dec 11 11:41:14 2003

Dear Tom,

Thanks for  your detailed explaination. I am also in favour of the Linux 
IP configuration. But it seems to take me sometime now to convince my 
boss to do that.

I will get back to you as soon as I got the feedback from my site.

Best regards, Mai