netcdf-hdf mailing list is no longer active. The list archives are made available for historical reasons.
Jeff, Thanks for sending the description of SILO. In general, I like your extensions to netCDF, and the way you managed to keep the new interface compatible with existing netCDF applications. I've provided tentative answers to your questions below, after some comments and questions of my own. If it's more convenient for you to just send me a copy of the detailed document rather than comment on all of these by email, that would be fine. First, it appears to me that the "directory" and "object" extensions are independent, and that the directory extensions are more fundamental. If object names are globally unique rather than scoped to directories (I couldn't determine this for sure from your description), than an object containing other components could be represented by a global attribute with a string value that listed the component dimensions, variables, and objects. For component names that differed from actual names, a syntax convention could be chosen to specify both names, e.g. including the component name in parentheses after the actual name. An attribute convention could be adopted that all such object attributes should have conventional names, for example beginning with the characters "Group_". Unless I'm missing something, I think it would be possible to write your object extensions as a small library that assumed such a set of attribute conventions. I'm not asserting this would be a better way to do it, only that it appears to be a possible implementation that wouldn't require interface extensions, only a new layer on top of the existing interface, and agreement about the attribute conventions. I'm convinced that adding the directory primitive requires the kinds of additions to the interface you have proposed. Directories would make some things we are proposing to do in the netCDF operators simpler, for example merging several input netCDF datasets into one output netCDF, but they also add some complexity to the interface. Since you've made the additions in an upward-compatible way, it seems that users could ignore this added complexity if they don't need the benefits it provides. If you don't create any directories, what name do you get back when you inquire about the name of the current directory? Is there a default directory id and name even when no calls to ncdirdef() have occurred? You wrote that there is still just one unlimited dimension ID. Is it possible to have multiple unlimited dimensions in different directories, but with the same ID? What is the "type" of an "object" that contains heterogeneous components, such as dimensions, variables, and other objects? Are object ID's unique within a dataset, like dimension ID's, or just within directories, like variables? Have you had a need for interfaces to add a new component to an existing object, to delete a component from an object, to copy or move a component from one object to another, or to iterate through the components of an object? Can the same component be in multiple objects? If objects were constructed with an "add" function, then you wouldn't have to specify the component type, but could instead use something like ncadddim() and ncaddvar() to add dimensions and variables to an object. Is it only possible to create directories and/or objects in "define mode"? Did you come up with an implementation that doesn't require distinguishing between "define mode" and "data mode"? Have you defined a Fortran interface for the new C functions in the interface? I notice you aren't on the netcdfgroup mailing list, so you may not know about the changes we made from int to long for the types of dimensions and hyperslab indices in netCDF 2.0. Is SILO based on netCDF 2.0 or an earlier version? Is the code for SILO available? Does your implementation require a commercial or proprietary database underneath SILO? Now here's my tentative answers to your questions: > 1. Should the concept of relative IDs be kept, even though it makes it more > difficult to specify object components? Should absolute IDs be > introduced, even though this departs from the netCDF model? It could be > possible to have both schemes simultaneously, and provide a mechanism > for mapping between absolute and relative. I think you made the right decision to change to relative IDs. With absolute IDs, you would need to add more functions to iterate through all the variables in a directory, for example. With relative IDs, when you are in a SILO directory you apparently cannot refer to components of other directories, but must always change directories first. With absolute IDs, It might be possible to get a dimension or variable from the closest ancestor directory that contains it if it is not found in the current directory. This would allow you to "factor out" common data without having to know exactly which parent directory contains it and to insert new intermediate levels in a directory hierarchy without breaking existing applications, so directories could be used for inheritance as well as name scope. I don't think this would be useful enough to justify changing from relative to absolute IDs. > 2. Is there a better name for objects than 'objects'? Perhaps 'groups'? "Object" has so many other connotations these days, that I would prefer a more specific name that had the connotation of being a collection. "Group" is fine, and some other possibilities include container, package, bin, bundle, etc. I'm CC:ing this reply to a local mailing list, netcdf-hdf@xxxxxxxxxxxxxxxx, of Unidata staff members interested in the NCSA project to put the netCDF (or SILO) interface on top of HDF. ---- Russ Rew russ@xxxxxxxxxxxxxxxx Unidata Program Center / U.C.A.R. P.O. Box 3000 Boulder, Colorado 80307-3000