[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Supporting other data formats

James Gallagher wrote:

On Apr 14, 2005, at 9:12 PM, John Caron wrote:

Tennessee Leeuwenburg wrote:

Hi guys,

We're looking at phase two of the project I'm working on at the moment. Phase one involved hooking up our weirdo database to OpenDAP, and it looks that that will be a success. There's one thing that's not as wonderful as we might hope, and a couple of cool extra things we'd like to do.

In order to cope with observational data stored in BUFR format, we've had to write scripts for conversion to NetCDF prior to serving it with thredds. I don't really like messing about in thredds code if I can help it, on account of it makes code maintainance that much harder. So we convert to NetCDF on a product-by-product basis. But it would be way cool if we could serve BUFR data more generally and without the conversion step. So my question is : does anyone on this list care much about BUFR data, and is anyone thinking much about how it might fit into an OpenDAP context?

What kind of data is stored in BUFR?

We have an "I/O service provider" framework in netcdf-java version 2.2, where you can read non-native files as if they are netcdf files.

This sounds really cool! Does this work by subclassing some generic access methods? This would make all sorts of one-off servers easy for us to write.


Hi James:

It uses delegation rather than inheritence, but its the same result. The "Service Provider" pattern is used extensively throughout the Java libraries, to allow you to "plug in" services like security, storage etc. In our case, a netcdf "I/O service provider" implements a small set of methods, which the NetcdfFile object uses whenever it does any I/O. The interface looks like:

public interface IOServiceProvider {
  boolean isValidFile( RandomAccessFile raf);

  void open( RandomAccessFile raf, NetcdfFile ncfile);

Array readData( Variable v2, List section);
// only if you use Structures
Array readNestedData( Variable v2, List section);


gory details as usual in the javadoc (see ucar.nc2.IOServiceProvider, implementations live under ucar.nc2.iosp) :


I am excited about the possibility of "one-off servers that are easy to write". This will allow both local access and remote access via opendap. I will write documentation (Real Soon Now) for anyone who wants to to write one of these, in the meanwhile, any "early adopters" can get special attention.

NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.