[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: NetCDF and HDF5

Sorry ted, your question started a discussion amongst us, and then i forgot to send you the results! 


Ted Habermann wrote:


John,
I wanted to check in with you and make sure that I understand the plan for the NetCDF > HDF5 project. I believe that, if I have data written in HDF5 I will be able to access those data using a NetCDF API and that existing code written with a NetCDF API (i.e. Ferret) will also be able to read the data. 

No, initially the only HDF5-format data that will be accessible
through the netCDF-4 API will be data written through the netCDF-4
API.  The HDF5 data model is more complex than the netCDF-4 data
model, and it will probably always be possible to create HDF5 files
that use features of HDF5 that are not supported by the netCDF-4 data
model.  In order to guarantee that all HDF5 data is readable via
netCDF-4, we would have to make netCDF-4 as complex as HDF5, and part
of the appeal of the netCDF interface is its simplicity/elegance.

After we have delivered the initial release of netCDF-4, we may try to
make more HDF5 data readable through the netCDF-4 interface while
preserving its simplicity, but there are complex features of HDF5 that
we would probably not want to try to support.



... Is it a reasonable assumption that if we write these data as
COARDS compliant HDF5 (whatever that means) the NetCDF apps will be
able to make sense of the data (if they are COARDS compliant as
well)?


If you write the HDF5 data through the netCDF-4 interface as COARDS
compliant (or preferably CF compliant), then netCDF-4 applications
will be able to access the data.



... Finally, what is a reasonable timeline for achieving this dream
world?


We're still on schedule to deliver the initial release of netCDF-4 by
June 2005, but that depends on NCSA developers delivering some needed
HDF5 enhancements in the third quarter of 2004 and our not
encountering any unanticipated problems.  If we don't finish the
NASA-funded project by June 2005, we could probably get a no-cost
extension from NASA, but that wouldn't fund additional development.

Please keep in mind that this project entails some risk.  Although
things look good so far, it's still possible that the trade-offs we
will need to make to support some requirements will be in conflict
with other requirements.  That's why we're thinking of this as
research, not just development ...

Now heres the tease:
We are exploring mapping HDF5 data model to an "extended Netcdf data model" so that many (most?) HDF5 files can be read through the netcdf-4 API. this is being prototyped in Java, with the possibility of being also done in C library, if it proves feasible. BUT neither the Java interface nor the transfer of Java technology to the C interface are funded by the current NASA grant, so will require extra resources. It may be that this functionality (if it works at all) will be only available in java. 
Sorry we cant be more sanguine about this work, but i'd hate for you to depend 
on something and then get let down.

john