Re: how often does HDF binary format change?

• To: netcdf-hdf@xxxxxxxxxxxxxxxx
• Subject: Re: how often does HDF binary format change?
• From: Quincey Koziol <koziol@xxxxxxxxxxxxx>
• Date: Sat, 25 Oct 2003 08:24:00 -0500 (CDT)

Hi Ed,

> > > Greetings to HDF HQ from NetCDFville.
> >     I see we're both early risers this morning... :-)
> 
> What kind of geek is programming at 6 am on a Saturday morning?
> 
> Our kind of geek, apparently!
    :-)

> > > I wonder how often you guys have to change the underlying binary
> > > format of the data in support of new releases of HDF5?
> >     We've tweaked the file format a little bit with each major release so 
> > far
> > (and sometimes with minor releases also).  This shouldn't be a problem for
> > applications which call the library though, since we provide full backward
> > compatability. (and as much forward compatibility as possible)
> 
> So you guys are committed to always support any HDF5 files?
    Yup.

> In 50 years, when we are old men, I will email you a file from last
> week, and ask you to read it with the then-current HDF library. Will
> you be able to?
    That's what we told NASA.  We are very committed to that promise and I
expect to fulfill it.
    We have made improvements to the file format and library which can cause
older libraries to not read newer files, but we have never changed the format
in a way to preclude reading old files with the library.

> > > And is there some format version number in the data file? I presume
> > > there is.
> >     We've learned that a single version number is not very useful, so we've
> > learned to include version #'s on all the structures inside the file (we 
> > missed
> > a couple, but I'm trying to correct that :-).  This "micro-versioning" has 
> > been
> > very useful and should allow us to avoid major revisions of the "entire" 
> > format
> > for the foreseeable future.
> 
> How interesting! That's a great idea. What, for example, are some of
> the structures that are versioned?
    If it's a block of values, it's got a version number (modulo the oversights
I mentioned).  So, we are versioning the file's superblock (which contains 
basic information about the size of offsets & lengths for the file, etc.), the
B-tree headers and nodes, the various structures used to represent groups, the
object header prologue, the object header messages themselves (which desscribe
an object's metadata like its datatype, dataspace, etc.), etc.  Ask John Caron
about this sometime... :-)

    Quincey