Hi Andrew, > Thanks for taking the time to send a personal note about this. We > (MINC) have been netcdf consumers for the past 15 or so years and HDF5 > for the past few. I suspect the misunderstandings in this post (to > minc-development) was more a bit of frustration over the varying > binary builds of HDF/netcdf that exist on various platforms. > > We build MINC across Windows (Cygwin), OSX and various flavours of > things ending in ux and ix. > > One of the things that tripped us up a lot was the builds of HDF 1.8.x > in Debian/Ubuntu with specific 1.6x compatibility flags but we are now > back on top of it. I HOPE! > > The main issue is that we now support both netcdf and hdf5 in MINC > (1.x and 2.x) and need a transition path out of this. This is > something that I would be most grateful to have your advice about. Due to a major refactoring of netCDF dispatch to support an OPeNDAP client for remote access, the variants of netCDF formats for classic and HDF5 storage, code to read some other formats data through the netCDF API, and a desire to support in-memory netCDF "files", we've temporarily broken the Windows Visual Studio port, however everything still builds with cygwin. We have plans to fix the Visual Studio port: http://www.unidata.ucar.edu/netcdf/docs/faq.html#windows_netcdf4_1 That may be a problem for you, and you may want to wait until the Visual Studio build works, if you use that. > My current feeling from what you have said is that we might be best to > drop HDF5 and instead use netcdf1.4 only. Our reasons for changing to > hdf5 was to take advantage of chunking, block compression and > arbitrary (and apparent) dimension ordering. > > From my quick reading this is all posible under netcdf1.4? NetCDF 4.1 supports chunking and block compression. I'm not sure what you mean by "arbitrary (and apparent) dimension ordering". NetCDF 4.1 supports multiple unlimited (extendable) dimensions, not just in different groups, but also for the same variable, so a multidimensional variable can grow incrementally and efficiently along multiple axes without copying data. We also got the HDF5 developers to add support for persistence of dimension creation order, so you can get things back in the same order you created them, if desired. However, I'm not recommending dropping HDF5. It still has features netCDF-4 doesn't support, and you would have to evaluate whether the power those features give you (even potentially if you aren't using them now) are worth the trouble of supporting multiple data formats. We have interoperability plans to increase support for HDF5 features, but we will probably always support only a subset of the features we regard as most useful, to keep the data model and API simple enough to understand. If we supported everything in HDF5, there would be little reason to continue to develop netCDF ... --Russ Russ Rew UCAR Unidata Program address@hidden http://www.unidata.ucar.edu Ticket Details =================== Ticket ID: AVF-863193 Department: Support netCDF Priority: Normal Status: Closed
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.