Re: Paper on OpenDAP
- To: Peter Cornillon <address@hidden>
- Subject: Re: Paper on OpenDAP
- From: Tennessee Leeuwenburg <address@hidden>
- Date: Fri, 12 Aug 2005 12:54:59 +1000
Yes I'd be interested in seeing your papers.
I'm in the middle of writing a proof-of-concept catalog crawler in
python. If that would be of general interest, I can keep the list posted
as to my progress.
It probably wouldn't be too hard to extract URLs from a google search
and test for whether or not they are catalogs. However, it might be
easiest to simply maintain a directory of catalogs via form submission
and support searching a crawled list...
We're looking here at trying to make a lot more use of opendap, so the
less wandering about in the Google forest, the better.
Peter Cornillon wrote:
I assume that James will chime in with some more info re plans in the
future. What I often do when looking for data is to use google. For
example, a Ferret user just asked the Ferret e-mail list where he
might get T/P data. I googled: topex opendap coards. Ferret feels
comfortable with COARDS so I figured that if I could find his data in
COARDS for available via OPeNDAP he should suck the data directly into
Ferret via OPeNDAP. I found the MERSEA site right off the bat. I've
used the same trick to find SST data and wind data. The work that we
are currently doing to incorporate THREDDS into OPeNDAP servers as
well as to upgrade our servers so that they indicate their presence on
the network is aimed at the same problem. Hopefully, one can use
Google or another search engine to find OPeNDAP servers and then one
can crawl these sites via the THREDDS catalog. We do have a project
with the UCSB Alexandria Digital Library group to work on better data
discovery and I believe that they are investigating a web crawler that
will look for OPeNDAP server based on some of the ideas that I have
brought forward re searches via Google. Hope that this helps.
p.s. If you are interested in papers that we have published re
OPeNDAP, please let me know and I will point you at them.
On Aug 11, 2005, at 8:34 PM, Tennessee Leeuwenburg wrote:
I have written a draft paper on the work I have been doing at the
Australian Bureau of Meteorology which has been accepted into a
conference here in September. In light of some of the reviewer
comments, I would like to spend a little more time describing the
OpenDAP community, and going into more depth w.r.t. XML data catalogs
One thing I thought I might try would be to write a catalog crawler
to demonstrate how one can discover data sources using automated
tools to a greater extent that possible under ad-hoc data publication.
I also thought I would get some feedback on what the community saw as
the most interesting aspects of thredds/opendap, and what new
directions are on the horizon (if I might be allowed to mix my
Graduate School of Oceanography - Telephone: (401) 874-6283
University of Rhode Island - Fax:
Narragansett, RI 02882 - E-mail:
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publicly available
through the web. If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.