Java netCDF-4 API

[John Caron]

Greg Rappa wrote:
> Hi John,
> 
> I work at Lincoln Lab, along with Oliver Newell.  We're working on a
> project, sponsored by the FAA, for standardizing the data formats
> and distribution methods of sensors and systems owned and operated
> by the FAA.  For several months now, I've been working on adapters
> that would convert data products from our home-grown weather system
> and distribute them via TDS or some other OGC-compliant
> catalog-based service (eg, some combination of CS/W and/or WCS).

Are these adapters following the nj22 "IO Service Provider" interface?

> 
> We have discussed our plans with Ben Domenico and Greg Thompson (et
> al) who share similar plans to distribute weather-related data
> products.  Our plans were to use the netCDF-Java version 2.2 library
> in order to generate HDF5 files.  The current nj22 release I'm using
> (2.2.18) still writes netCDF-3 format files.  We were hoping to
> generate netCDF-4 files instead to take advantage of the underlying
> HDF5 disk format, particularly in support of the size of our
> national data products.

We will eventually support writing netcdf-4 files, but the time frame is unclear. Im hoping to have something in 3 months, but cant guarantee it. In the meanwhile, have you experimented with writing netcdf-4 with the C library? What kind of file sizes do you see over netccdf-3? File reading/writing slower or faster? Can you send me any sample netcdf-4 files?

> 
> Our understanding is that the Java API for writing NetCDF-4 files is
> in the works, and we are trying to figure out the right road to take
> to meet our project goals.  We actually have some flexibility
> with respect to the order we do things, and we are interested in
> contributing back to the community if there is something (that isn't
> too hard!).  We'd be interested in a brief telecon if you could
> spare a moment (anytime between 3:00 - 5:30 EST today, or pretty
> much anytime tomorrow, or sometime next week if that's better for
> you).

I will be out for most of next week, but the following week may be possible. Lets use email to clarify the issues for now.


> 
> We went to the OGC meeting in Ottawa last month, and have a few
> thoughts on that, as well, if you are interested.  For example,
> there are some scaling & complexity issues that result from the fact
> that there are separate WFS/WCS services as opposed to one service
> that understood features (vector data) AND coverages (raster data)
> via a set of configurable plug-ins.  Maybe it's too late and the OGC
> train has left the station, but we'd be interested in your opinion
> on that topic.

Not that I understand it very clearly, but OCG wants to make features the root of their object hierarchy, with coverages a kind of feature. We recently met with British Atmos. data Center to examine ways to evolve and converge both OGC and netcdf data model. So lots of people are interested in further evolution.

But, I dont know how the political OGC and web services winds might shift. We are also supporting REST web services as well as OPeNDAP. We will be also be upgrading to WCS 1.1 sometime this year. So we are keeping our money spread out over several horses... 

> 
> Thanks,
> Greg.
> 
> |---------------------------------------------------------------
> Greg Rappa
> MIT Lincoln Laboratory               Group 43 - Weather Sensing
> Phone: 781-981-3660, Fax: 781-981-0632, Email: gregr at ll.mit.edu
> |