NOTE: The netcdf-hdf mailing list is no longer active. The list archives are made available for historical reasons.
Hi Ed,
> Naturally you're more familiar with the HDF5 interface than I am. I've
> read your documents and just have a couple of comments/questions:
>
> 1 - I gather you are using a time-stamp instead of an index. Have you
> considered what will happen if two different machines add a
> dataset/variable to the same netcdf-4 file, and their clocks are
> wrong? Or lets say I change the time on my machine and then open the
> file and add a variable. Did I just break the creation order?
Yes, you could "break" the creation order in these situations. There are
a number of advantages to using a timestamp instead of an index - I'll try to
talk about them in my next e-mail...
> 2 - I will never need to go backwards through the list. When I read an
> unknown file, I read in all metadata in one pass into internal
> netcdf-4 structs. So I only need to go through each group in forward
> order.
Ok.
> 3 - As I understand it, this applies on a group by group basis. That
> is, I will be able to open a group, and cycle through it's objects in
> their creation order. Then I can open another group and find *its*
> objects, in creation order. Is that correct?
Yes, the creation order information is maintained on a "local" basis.
> 4 - I don't use H5GIterate at all, so it doesn't have to be modified
> for me. Below I show the code I use. It's in nc4hdf.c, as are most of
> the HDF5 calls in netcdf-4. (Pertinent code is included below).
Sure, that'll work also. However, it makes sense to update H5Giterate
(or make a new API routine) to take advantage of the changes proposed.
> 5 - From the "group access" property list part, I'm getting the idea
> that I can write a HDF5 file with a bunch of datasets, then open it,
> and by opening the group with the proper "group access" property, I
> can get the objects in creation order. In other words, I don't have to
> take any special effort in creating the HDF5 file, the creation order
> is stored always. Is that correct?
No, the additional index (or indices, at some point) are only created when
the group is created and need to be specified in the group creation property
list when the group is created. The group access property list will allow you
to choose which index (if there is more than one) to use for the "by_idx"
API routines.
> 6 - Here's the code I have now. As I understand your design, I will
> just have to open the group with the "group access" property list and
> this code will work fine, because H5Gget_objtype_by_idx will return
> objects in creation order.
As I mention above, you'll first have to create the group with an index
that tracks the creation time field of the links in it (unless we decide to
default to always creating that index, but I don't think that will be the case),
and then specify the creation time index as the index to use for the "by_idx"
calls that you use. If you do that, your code below looks fine.
> if ((grp->hdf_grpid = H5Gopen(locid, grp->name)) < 0)
> BAIL(NC_EHDFERR);
> }
>
> /* Find the variables. Read their metadata and attributes. */
> if (H5Gget_num_objs(grp->hdf_grpid, &num_obj) < 0)
> BAIL(NC_EVARMETA);
> for (i=0; i<num_obj; i++)
> {
> /* Get the type (i.e. group, dataset, etc.), and the name of the
> object. */
> if ((obj_type = H5Gget_objtype_by_idx(grp->hdf_grpid, i)) < 0)
> BAIL(NC_EHDFERR);
> if (H5Gget_objname_by_idx(grp->hdf_grpid, i, obj_name,
> NC_MAX_HDF5_NAME) < 0)
> BAIL(NC_EVARMETA);
> LOG((4, "Encountered: HDF5 object obj_type %d obj_name %s", obj_type,
> obj_name));
>
> /* Deal with groups and datasets. */
> if (obj_type == H5G_GROUP)
> {
>
>
> 7 - What about attributes? Can I get them by creation order too? You
> make no mention of H5Aopen_idx, but that's what I use. Here's the code
> I use to read atts in a group:
>
> num_obj = H5Aget_num_attrs(grp->hdf_grpid);
> for (i = 0; i < num_obj; i++)
> {
> if (attid > 0) H5Aclose(attid);
> if ((attid = H5Aopen_idx(grp->hdf_grpid, (unsigned int)i)) < 0)
> BAIL(NC_EATTMETA);
> if (H5Aget_name(attid, NC_MAX_NAME + 1, obj_name) < 0)
> BAIL(NC_EATTMETA);
> LOG((4, "reading attribute of _netCDF group, named %s", obj_name));
As I mentioned during today's telecon, as long as attributes aren't
deleted from objects, this should be fine.
Quincey
netcdf-hdf archives: