Re: 1.8 file format

[Quincey Koziol <koziol@xxxxxxxxxxxx>]

Hi John,

On Jul 11, 2007, at 6:24 PM, John Caron wrote:

> Quincey Koziol wrote:
> > Hi John,
> > On Jul 2, 2007, at 5:16 PM, John Caron wrote:
> > > Hi Quincey:
> > >
> > > Any idea when you will get to this?
> >     Yes, one of the HDF5 developers here (Vailin Choi) is wrapping  
> > up some work on the 'h5stat' tool, which will probably take  
> > another 2-3 weeks.  Then, she'll start updating the file format  
> > spec. and adding support for the new format changes to the  
> > 'h5check' tool (which validates that an HDF5 file conforms to the  
> > format spec.).  So, in about a month, we'll be able to start  
> > feeding you updates to the file format spec.
> >     If you'd like to start earlier than that, I can guide you to  
> > the relevant portions of the source code, which are fairly  
> > isolated and reasonable well documented.
>
> ok, that sounds good, how do i start?

	Hmm, let's start with the updated superblock format, then the new  
object header format and new/updated object header messages and then  
we can proceed to the v2 B-trees and the fractal heap.  My discussion  
below uses line #'s in the hdf5-1.8.0-beta2 release that we put out  
two weeks ago.  You can create a file that uses all new format  
objects by setting the "latest format" file access property when the  
file is created.

	So, for the changes to the superblock, you'll need to look in src/ 
H5Fsuper.c at the H5F_super_read() routine.  Starting on line 262 you  
can see that the 'p' variable is used to walk through the buffer  
containing the superblock.  First the fixed size portion is parsed,  
then the code splits to read the older version of the superblock  
(which might be worth a read to see if it aligns with your code) and  
then the new version of the superblock is parsed.  A short while  
after that, the code checks for a superblock extension.  The  
superblock extension is a more flexible way to add more information  
to the file's superblock, without forcing the superblock format to  
keep changing.  The superblock extension is stored as an object  
header, with new pieces of superblock information stored as object  
header messages.

	Let me know what your questions are about this section of code and  
we'll proceed from here.

	Quincey

Attachment: smime.p7s
Description: S/MIME cryptographic signature