[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Paper on OpenDAP

1st- we here at NOMADS also need the BUFR interface, and as some might recall, we went round-and-round on this topic about 6+ months ago.  We developed our own BUFR decoder and placed them into GrADS station files to access via our OPeDNAP.  It's messy tho' and some data (and the BUFR itself) does not work (we noted that we found some BUFR has compression flags turned on yet- they are not compressed!) A real nightmare....anyway

So-I'd do all I can to push for these (CF and COARDS) conventions- but it will take time for our community to
1) adopt and fund these efforts of data models
2) instruct scientists and data managers on just how to go about placing their data into these conventions. 

As I'm a tad bit out of my domain here- a question: howabout the Common Data Model at Unidata?  How will this fit in?  Maybe that's the driver we need to converge or nudge our community?  Glenn

btw Tennesse:  our NOMADS links are broken (I'm collaborating w/ Neville). 
We had
  Products page: http://www.bom.gov.au/nmoc/

Do you have updates....Thx, Glenn

Tennessee Leeuwenburg wrote:
Just some off-the-cuff responses.

The GODAE project is worth looking into here -- they have adopted OpenDAP and CF. I don't know how much activity the project has. There seems to be a difference between USGODAE and the vanilla flavour. At any rate, they don't seem to have published anything of significant for a couple of years, but the project is a really nice idea.

Here in the Bureau, we have a moderate degree of standardisaton on COARDS, but it's not an absolute rule.

The "easiest" way might be to support some particular convention in the output, with the server being able to convert to that from a number of different input conventions. For example, write code that will take either a COARDS or CF input file, and do the relevant modifications to the semantic-structure to present consistent output. That at least would greatly simplify the task of visualisation (for example). If coupled with a good generic data viewer (like IDV), then people would me even more inclined to adopt the standard.

Here at the Bureau the biggest obstacle is the lack of a BUFR interpreter for OpenDAP / thredds. This is a big deal for us, and we're going to have to go with a hybrid solution instead of using OpenDAP across the board. BUFR is an acronym for Binary Universal File Representation, and has been adopted by the World Met Organisation as a standard for obvervational data. It's a complete hassle to deal with, because the level of software support is low.

The other thing I thought would be cool (while we're throwing ideas around like popcorn) would be if Thredds was also a web service, with a WSDL and so forth.

Do any such conventions exist for OpenDAP?

No. The idea is to let different disciplines impose whatever structure they want on top of OPeNDAP. The more appropriate question would be "Does the oceanographic (or the Earth Science) community have any such convention?" Unfortunately, the answer is again no. That is my fault. I should have recommended one a long time ago. I think that COARDS (or CF) would be an excellent starting point and COARDS is becoming a de facto standard in that many data providers use it. I was reluctant to impose COARDS on the oceanographic use of the system since that would mean the reordering of some archives which could be very costly. I prefer a modified C
COARDS, one that does not adhere to the semantic structure, but does adhere to the rest of the standard.

Here at BOM we more or less have our conventions agreed on for my particular project, but if there were any recommended best practise it could be interesting.

What do you think of adopting COARDS or CF? Would one or the other address all of your data sets? If not, what is missing? If it could, but your would rather not adopt it, it would really be useful to know why and to know what you guys have adopted and why. I am hoping that the Marine Metadata effort comes up with a recommendation for the oceanographic community. I'm not sure what is being done for the meteorological community, but it would be great if there was a similar effort (to the Marine Metadata program) and if they came up with a recommendation.



 Peter Cornillon
   Graduate School of Oceanography     -  Telephone: (401) 874-6283
      University of Rhode Island                 -               Fax: (401) 874-6728
        Narragansett, RI 02882                    -           E-mail: address@hidden

Glenn K. Rutledge
NOMADS Program Manager
NOAA Meteorologist / Physical Scientist
National Oceanic and Atmospheric Administration
National Climatic Data Center
151 Patton Ave
Asheville, North Carolina 28801
(828) 271-4097

The contents of this message are mine personally 
and do not necessarily reflect any position of the 
Government or the National Oceanic and Atmospheric Administration."

NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.