Russ is a very patient product manager. Some would fume over delay, some would rant and rave, but Russ doesn't do any of those things. 

Last December, when I told Russ it would take a few weeks to get 4.1.2 ready for release, he visited me in the coding cave and brought some supplies. As he rolled the boulder across the entrance, he called out that he had actually provided two months of supplies. I think that we can all agree that this was above and beyond the call of duty.

In fact, Russ underestimated what could be done with flour and baking powder, and those two months of supplies lasted until the beginning of March. Around that time, after the rat attack, I also ran out of candles. Fortunately the release was almost ready. After it was complete, Russ not only arranged to have the boulder rolled away, he was even waiting outside with a meal and some medical attention (for the rat bites).

See the netCDF web page for information about the new release, and to get a copy.

I hope to do releases a lot more frequently in the future. The 4.1.3 release should come in about 3 months. I love my coding cave, but I also like to see the sky at least three or four times a year. That means a branch of the code at the end of May, with hopefully the following features:

• refactored and much simpler build system (almost done on branch ed).
• netCDF disk-only files (to support gridspec and opendap).
• support for the new super-large format developed by the parallel-netcdf team.
• more hdf5 interoperability tweaks.
• start to switch documentation to doxygen.
  
• documentation fixes.
• upgraded libcf distribution.

Windows DLL versions of the netCDF C library have usually come last in our list of supported platforms. This is not only because it was uncommon among science users, but also because we mostly don't develop in windows, and it's significantly different.

As a result we released DLLs for windows for almost all our versions, but this system broke down after the 4.0 release. Since then, we have not done a windows DLL release, and that is too long.

To try and bring windows into the mainstream of netCDF testing and development, we are now using Cygwin, with mingw32 libraries, to build netCDF DLLs.  This is something I just got working last week. The results can be seen on the daily snapshot page (you must be logged into the Unidata website to view this page):

https://www.unidata.ucar.edu/software/netcdf/builds/snapshot/

The builds on Carson are on a Cygwin machine, and the ones with description field "no OPeNDAP DLL build on Cygwin" are the DLL building tests. (The other tests on Carson are just straight Unix builds on Cygwin.)

The current results show that this is working for the non-netCDF-4 version of the DLL. But what do I mean by "working?" All tests pass, and I am left with a DLL, so that's good. But the DLL is not the whole story, Microsoft developers also need a .lib file, which describes the DLL. There is a way to produce this with my daily build too, and I have to get it working.

There is also the problem that the DLL is named cygnetcdf.dll, and it should be named netcdf.dll. A simple rename, but I have to look into why the tools put the "cyg" there in the first place, to see if it is important.

But, though much work remains to be done, I am happy to be producing a netCDF DLL as part of my daily snapshot. All this is much easier than it used to be, due to steady improvement in Cygwin and mingw32. To build this DLL, the configure must be run with options --enable-dll and --target=mingw32.

The next step is to get an automated binary release of the DLL, so that I can easily get it out to the users. In the meantime, if anyone would like to give it a try, email me and I will send it. This is the C API only, without netCDF-4 or OPeNDAP.

The end goal is to produce netCDF-4 and OPeNDAP capable DLLs, as part of the normal build process, and with the same autoconf/automake tool set we use for all other platforms.

How do I love Valgrind? Let me count the ways.

Valgrind is a neat little tool that replaced the memory handling routines of the operating system with specially instrumented ones that also keep track of everything you are doing with memory. Then, if you don't free it, Valgrind can tell you.

All of this will seem unspeakably primitive to our Java programming friends. Sorry to bring up such a barbaric topic as memory management.

Like any such tool, when Valgrind was first used on netCDF code it issued many warnings and error reports. Most were actually warnings and memory errors in the test programs themselves (which don't get the kind of attention that the library code does - who tests their tests?) But some of the Valgrind messages pointed to real memory bugs in either HDF5 or netCDF-4.

The HDF5 team has been very pro-active in hunting down all the memory problems this process has uncovered, and since 1.8.4 have been tightening up memory handling by HDF5. Meanwhile I have been doing the same for netCDF-4 code.

The result is that (in my branch of the repository - soon to be merged into the main branch) there are very few memory leaks of any kind, and almost all the libsrc4 test programs now pass Valgrind with no errors or warnings. These changes will be part of the performance and bugfix release 4.1.2.

I love Valgrind because all previous tools I've used for this have been rather clumsy. Valgrind is the easiest way to memory test a program!

The NPP satellite mission is to produce data in HDF5. There is great interest in seeing these data through the netCDF API, so that netCDF tools can work with NPP data.

At the HDF5 workshop this week I have been given a sample file by Elena which is like the NPP files we will eventually see.

NetCDF can read HDF5 files as long as they follow certain rules, but the NPP files doesn't all those rules. In particular, they use the reference type, currently no handled by netCDF. My plan is to have netCDF not give up when it runs into a reference object.

Now we wait for any bugs to be reported...

It is always a tremendous amount of effort to make a netCDF release. There are so many people depending on it, and I really don't want to mess up up.

The 4.1 release, which I did last Friday, was my sixth netCDF release. This release has a lot of new features - more than usual, since we have had more than the usual number of programmers working on netCDF. It hasn't been just me and (part-time) Russ, as in previous releases. This is the first release that contains work from Dennis, and what a large piece of work it does contain: the new opendap client.

In addition, the 4.1 release contains a new utility, nccopy, which copies netCDF files, changing format on the way if desired. (Russ did nccopy). There is also a new ncgen, finally up to speed on all netCDF-4 extensions, which Dennis did.

As for me, I put in a layer that can read (many but not all) HDF4 files, a layer that can use parallel-netcdf for parallel I/O to classic/64-bit offset files, a modification to allow most existing HDF5 data files to be read by netCDF. Finally, and at the last minute, I changed the default settings of caching and chunking so that the users producing the giant AR-5 datasets would get good performance. Also, I added the nc-config utility to help users come up with correct compiler flags when building netCDF programs, and two new libraries are part of the distribution, UDUNITS (Steve Emmerson) and libcf (yours truly).

Probably we should have released 6 months ago, and held some of those features back for another release. The more new features, the harder it is to test and release.

But it's out now, and we have started working on release 4.2. For that release we have more limited ambitions, and I hope it will be out this year.