Re: Java netCDF-4 API

• To: Greg Rappa <gregr@xxxxxxxxxx>
• Subject: Re: Java netCDF-4 API
• From: John Caron <caron@xxxxxxxxxxxxxxxx>
• Date: Thu, 31 May 2007 16:24:57 -0600

Greg Rappa wrote:

Hi John,

I work at Lincoln Lab, along with Oliver Newell.  We're working on a
project, sponsored by the FAA, for standardizing the data formats
and distribution methods of sensors and systems owned and operated
by the FAA.  For several months now, I've been working on adapters
that would convert data products from our home-grown weather system
and distribute them via TDS or some other OGC-compliant
catalog-based service (eg, some combination of CS/W and/or WCS).

Are these adapters following the nj22 "IO Service Provider" interface?

We have discussed our plans with Ben Domenico and Greg Thompson (et
al) who share similar plans to distribute weather-related data
products.  Our plans were to use the netCDF-Java version 2.2 library
in order to generate HDF5 files.  The current nj22 release I'm using
(2.2.18) still writes netCDF-3 format files.  We were hoping to
generate netCDF-4 files instead to take advantage of the underlying
HDF5 disk format, particularly in support of the size of our
national data products.

We will eventually support writing netcdf-4 files, but the time frame is 
unclear. Im hoping to have something in 3 months, but cant guarantee it. In the 
meanwhile, have you experimented with writing netcdf-4 with the C library? What 
kind of file sizes do you see over netccdf-3? File reading/writing slower or 
faster? Can you send me any sample netcdf-4 files?

Our understanding is that the Java API for writing NetCDF-4 files is
in the works, and we are trying to figure out the right road to take
to meet our project goals.  We actually have some flexibility
with respect to the order we do things, and we are interested in
contributing back to the community if there is something (that isn't
too hard!).  We'd be interested in a brief telecon if you could
spare a moment (anytime between 3:00 - 5:30 EST today, or pretty
much anytime tomorrow, or sometime next week if that's better for
you).

I will be out for most of next week, but the following week may be possible. 
Lets use email to clarify the issues for now.

We went to the OGC meeting in Ottawa last month, and have a few
thoughts on that, as well, if you are interested.  For example,
there are some scaling & complexity issues that result from the fact
that there are separate WFS/WCS services as opposed to one service
that understood features (vector data) AND coverages (raster data)
via a set of configurable plug-ins.  Maybe it's too late and the OGC
train has left the station, but we'd be interested in your opinion
on that topic.

Not that I understand it very clearly, but OCG wants to make features the root 
of their object hierarchy, with coverages a kind of feature. We recently met 
with British Atmos. data Center to examine ways to evolve and converge both OGC 
and netcdf data model. So lots of people are interested in further evolution.

Thanks,
Greg.

|---------------------------------------------------------------
Greg Rappa
MIT Lincoln Laboratory               Group 43 - Weather Sensing
Phone: 781-981-3660, Fax: 781-981-0632, Email: gregr@xxxxxxxxxx
|