--
Ethan R. Davis Telephone: (303) 497-8155
Software Engineer Fax: (303) 497-8690
UCAR Unidata Program Center E-mail: edavis@xxxxxxxx
P.O. Box 3000
Boulder, CO 80307-3000 http://www.unidata.ucar.edu/
---------------------------------------------------------------------------
--- Begin Message ---
Hi Peter,
The THREDDS catalog XML was designed to be used to describe and provide
access information for pretty much any data sets. The access information
for a dataset is a URL and an access/service type. The metadata that can
be included in a catalog is very flexible. We have defined a basic level
of metadata but, also, any XML can be contained in or a URL can be
referenced by the <metadata> element.
The THREDDS Data Server (TDS) can serve data via HTTP and OPeNDAP and
does automatic catalog generation for the data it is serving. It can
also catalog data served by other servers. It can serve any type of data
over HTTP. For a dataset to be served via OPeNDAP, it must be readable
by the netCDF-Java library. The netCDF-Java library (ver 2.2) can
currently read netCDF-3 files, HDF5, GRIB, GINI, NEXRAD radar data
(level 2 and 3), as well as data served by OPeNDAP and ADDE servers.
If you have other types of data and want them served via OPeNDAP, the
netCDF-Java library does provide an interface that can be implemented
for new data formats allowing them to be read by the netCDF-Java library.
The TDS does not come with any large scale or cross-catalog
search/discovery service. We do use our DQC documents in a number of
places. We mainly use DQCs (Dataset Query Capability) for requesting a
subset of a homogeneous dataset collection, for instance, the last hour
of radar data from a certain station. We have been thinking some about
extending the DQC for more extensive discovery but that is still in the
thinking stage. We work pretty closely with a group that crawls a
network of THREDDS catalogs to gather up metadata information and
provide a search service on that collection of datasets. We are also
working with some other discovery service providers to feed them our
metadata records.
Hope that helps. Let me know if you have any other questions.
Ethan
Peter Mooney wrote:
Hello there,
My name is Peter Mooney and I am currently working with the
Environmental Protection
Agency in Ireland. I am working on IT Systems Development. I subscribe
to the THREDDS mailing list
and am currently begining to dabble in the world of THREDDS and
netCDF. I apologise for sending an email directly to you guys but I
don't want to send it to the entire list as it may be off the point of
the THREDDS discussion.
A small overview of background to this project.
EPA-Ireland invests significantly in research programmes in MSc and
PhD programmes in
universities and colleges in Ireland. In the past the data and results
that were
generated from these research projects were effectively lost in time -
the results were
published in journals but the data was not archived. In my work group
we have
build a web-based system to allow funded projects to "upload" data (by
FTP and HTTP) to our
SAN server for safe archive and storage of these datasets and results.
The EPA-Ireland
have committed to safe-guarding this information going forward. Many
of the datasets
generated are of interested to other researchers, stakeholders, the
general public etc.
As you will agree this is where the real problems start - how to
standardise datasets
coming in from many different projects/systems/locations. This is
compounded by the fact that
they have to be archived and made available to allow as wide a range
of stakeholders to
access them.
We have now begun to look at providing web services such as THREDDS.
We are working with Tomcat/Apache/Servlets etc in our everyday work so
we are comfortable
with working with this excellent system.
We have a few questions specific to our work in relation to THREDDS -
I wonder could
I be so bold as to ask them? I understand how busy you guys are and
with the holiday season
approaching and all. I would be very grateful of any information.
We are considering a process of converting datasets we have in our
archive to netCDF
and then using THREDDS to provide these datasets in a Web Service
architecture.
However given the variety and 'quirkiness' of some of the data this
could be a difficult
task. Is it possible to use THREDDS such that it can provide a Web
Service to datasets
not in netCDF format? Could we use it to index our datasets in their
original format?
We would like users to be able to get listings of our data holdings
and then retrieve datasets
that match their specific queries (queries based on the metadata
associated with each data resource)
The team I work in (3 in total) are not concerned with having to roll
up the sleves and do some
software dev work to make THREDDS work in the way described above.
Apologies if I have not articulated my ideas in a coherent manner - I
would be happy to re-write my
ideas if you should require it.
I look forward to your reply,
Thank you again for your time,
Best wishes and have a nice day,
Peter Mooney
------------------------------------------------------------------------
Play Santa's Celebrity Xmas Party
<http://santas-christmas-party.yahoo.net/>, an exclusive game from Yahoo!
--
Ethan R. Davis Telephone: (303) 497-8155
Software Engineer Fax: (303) 497-8690
UCAR Unidata Program Center E-mail: edavis@xxxxxxxx
P.O. Box 3000
Boulder, CO 80307-3000 http://www.unidata.ucar.edu/
---------------------------------------------------------------------------
--- End Message ---
