[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

19990513: survey results....LDM data recovery (part 1)



>From: Peter Neilley <address@hidden>
>Organization: NCAR/RAP
>Keywords: 199905131803.MAA21500 archive data access

Peter,

I read with great interest your reply to David W.:

>This is indeed what we are currently setting up here at NCAR with 
>some NSF funding.  Our IITA server (which most sites use as a
>failover when the top-tier site thelma is down) will collect and
>store a recent archive of feedtype files, in nearly the same manner.
>(Perhaps we should do it exactly the same manner, I'll contact
>David to discuss this).  We planned to make this archive available
>via a web/ftp interface.  Details will follow

There is also a new project underway between NCAR/SCD and the UPC.
The short term goal is to provide access to recent data received
on the IDD free-of-charge to Unidata sites.  FTP access will be the
first method followed by some sort of web interface.  This goal
sounds exactly like yours.

A longer term goal is to provide access to the NCAR Mass Store to
the Unidata community, again free-of-charge.

Since there are several organizations doing/in the process of
doing/planning on doing the same sort of thing, we ought to collaborate
on access methodologies, etc..  As you may imagine, the UPC finds
itself short on manpower right now, so we are looking to leverage
whatever we can from those doing development for the greater community
good.

Any input that you/RAP can have on this would be extremely valuable.
In particular, I am very interested in:

o how you plan on storing the data (e.g. hourly files by feed type as is
  done at U. Illinois and U. Washington)
o what ideas you have for the web interface you mentioned
o what data you are most likely to save
o how recent your "recent archive" would be (e.g. 5-7 days; two weeks, etc.)
o the form of the data (e.g. raw, GEMPAK format, etc.)
o any thoughts on storing the "recent archive" to tape and access to
  that data
o any thoughts on running a data server that could slice and dice the
  data for the user (e.g. DODS, McIDAS ADDE, etc.)
o anything else

Right now, SCD has made 8 GB of disk and an account on the new
'dataproc' compute server available to us.  We have already loade the
LDM on the machine and will begin testing feeding IDD data to it soon.
In three months, our disk storage will be increased to 100 GB.

Al Kellie (head of SCD) is very interested in making this joint
collaboration successful.  In our initial planning meeting he noted
that providing access to the NCAR Mass Store holdings was not only good
for the Unidata community, but it will be good for SCD as well.

In short, we must not let the opportunity to share
ideas/development/experience pass us by.

Any input you can provide us would be greatly appreciated.

Tom
--
+-----------------------------------------------------------------------------+
* Tom Yoksas                                             UCAR Unidata Program *
* (303) 497-8642 (last resort)                                  P.O. Box 3000 *
* address@hidden                                   Boulder, CO 80307 *
* Unidata WWW Service                             http://www.unidata.ucar.edu/*
+-----------------------------------------------------------------------------+