[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Paper on OpenDAP



Hi Peter,

Yes I'd be interested in seeing your papers.

I'm in the middle of writing a proof-of-concept catalog crawler in python. If that would be of general interest, I can keep the list posted as to my progress.

It probably wouldn't be too hard to extract URLs from a google search and test for whether or not they are catalogs. However, it might be easiest to simply maintain a directory of catalogs via form submission and support searching a crawled list...

We're looking here at trying to make a lot more use of opendap, so the less wandering about in the Google forest, the better.

Cheers,
-T


Peter Cornillon wrote:

Hi Tennessee,

I assume that James will chime in with some more info re plans in the future. What I often do when looking for data is to use google. For example, a Ferret user just asked the Ferret e-mail list where he might get T/P data. I googled: topex opendap coards. Ferret feels comfortable with COARDS so I figured that if I could find his data in COARDS for available via OPeNDAP he should suck the data directly into Ferret via OPeNDAP. I found the MERSEA site right off the bat. I've used the same trick to find SST data and wind data. The work that we are currently doing to incorporate THREDDS into OPeNDAP servers as well as to upgrade our servers so that they indicate their presence on the network is aimed at the same problem. Hopefully, one can use Google or another search engine to find OPeNDAP servers and then one can crawl these sites via the THREDDS catalog. We do have a project with the UCSB Alexandria Digital Library group to work on better data discovery and I believe that they are investigating a web crawler that will look for OPeNDAP server based on some of the ideas that I have brought forward re searches via Google. Hope that this helps.

Peter

p.s. If you are interested in papers that we have published re OPeNDAP, please let me know and I will point you at them.


On Aug 11, 2005, at 8:34 PM, Tennessee Leeuwenburg wrote:

Hi all,

I have written a draft paper on the work I have been doing at the Australian Bureau of Meteorology which has been accepted into a conference here in September. In light of some of the reviewer comments, I would like to spend a little more time describing the OpenDAP community, and going into more depth w.r.t. XML data catalogs etc.

One thing I thought I might try would be to write a catalog crawler to demonstrate how one can discover data sources using automated tools to a greater extent that possible under ad-hoc data publication.

I also thought I would get some feedback on what the community saw as the most interesting aspects of thredds/opendap, and what new directions are on the horizon (if I might be allowed to mix my metaphors).

Cheers,
-T

--
 Peter Cornillon
   Graduate School of Oceanography     -  Telephone: (401) 874-6283
University of Rhode Island - Fax: (401) 874-6728 Narragansett, RI 02882 - E-mail: address@hidden