Updated group API changes draft

Hi all,
    I've spent some time updating my previous draft of changes to the group API
as a result of the changes necessary to provide "creation order" support for
users.  I've attached the original version and the updated 2nd draft below.
    Please send me feedback, as I'll be starting coding on this soon.

    Quincey
NOTE #1: These changes are mainly ramifications from the addition of an
    [optional] index that allows applications to access objects in a group
    according to the order they were created.

NOTE #2: We still need to decide on scheme for re-naming existing functions
    when their parameters change.

Existing API routine changes:
============================

Old Name:               Purpose:                Change:
=========               ========                ======
H5Gcreate               Create new group        Will need to drop 'size_hint'
                                                parameter and acquire "group
                                                creation" and "group access"
                                                property lists

H5Gopen                 Open an existing        Add "group access" property
                        group                   list

H5Gclose                Close a group ID        <none>

H5Gset_comment          Set a "comment" on a    <none> (deprecated)
                        group

H5Gget_comment          Get the "comment" on a  <none> (deprecated)
                        group

H5Glink/H5Glink2        Creates a link to an    <none>
                        object in another group
                        by name in the other
                        group

H5Gmode/H5Gmove2        Renames a link to an    <none>
                        object in current group
                        by name

H5Giterate              Iterate through objects Change to allow choice of which
                        in a group, with user   index to use (and direction?).
                        callback routine

H5Gget_objinfo          Retrieve information    Update H5G_stat_t with any new
                        about an object in a    information.  Possibly add
                        group                   H5Gget_objinfo_by_idx routine?
                                                Absorb H5Gget_<foo>_by_idx
                                                routines?

H5Gget_num_objs         Retrieve number of      <none>
                        objects in a group

H5Gget_objname_by_idx   Retrieve name of n'th   Needs a "type of index"
                        object in a group,      parameter, in order to choose
                        according to link name  index to use to compute the
                        order                   "n'th object" of.

H5Gget_objtype_by_idx   Retrieve type of n'th   Needs a "type of index"
                        object in a group,      parameter, in order to choose
                        according to link name  index to use to compute the
                        order                   "n'th object" of.

Added API routines:
==================

Name:                   Purpose:
=========               =======
H5Glink_order           Create a link to an object in another group by creation
                        order in the other group.

H5Gunlink_order         Remove the n'th object (by creation order) in a group.

H5Gchange_time          Change the creation time of an object in a group,
                        potentially changing its position in creation order.

H5Gget_linkval_by_idx   Retrieve the link information about a soft link, using
                        the order with an index.

H5Gopen_by_idx          Open the n'th object in a group, accordding to
                        the order with an index.

New property lists:
==================

Name:                   Purpose:
=====                   =======
"Group Creation"        Control various parameters of a group, like it's B-tree
                        properties and whether it has an index allowing it
                        track the order that objects are created.  Possibly
                        other parameters as well.

"Group Access"          Control various parameters of accessing a group, like
                        possibly the default index to use for various
                        operations, etc.
NOTE #1: These changes are mainly ramifications from the addition of an
    [optional] index that allows applications to access objects in a group
    according to the time the object (or link) was created.

NOTE #1a: We will track creation order as the offset into an index on the
    creation time of an object/link.  The creation time of an object/link will
    be encoded as three parts: the number of seconds since the UNIX epoch when
    the object was created (as a signed 64-bit number), the number of
    nanoseconds since the second (as an unsigned 32-bit number) and a "counter"
    field (as an unsigned 32-bit number).

    The counter field is needed due to the lack of actual nanosecond
    granularity of most system clocks which could allow objects to appear be
    created at the same second/nanosecond.  If the same second/nanosecond
    would be used to create a new link, the counter value will be incremented
    until a unique second/nanosecond/counter triplet is arrived at.

NOTE #1b: We could create create a little API for these unique timestamps, that
    used the algorithm above.  This might be nice for users who want to
    timestamp various events that occur.  However, we will have to do some
    additional work to make this work in parallel.  Additionally, this may be a
    problem for us, since users will want to store these in a file and would
    assume that we should provide a datatype for them, etc.

NOTE #2: If a multiple indices are active in a group when objects are created
    in that group (with H5Dcreate(), etc.), they will automatically be updated
    to include the newly created object.

NOTE #3: We still need to decide on scheme for re-naming existing functions
    when their parameters change.


=====================
Existing API routines:
=====================

--------------------------------------------------------------------------------

Current Name:
    H5Gclose

Prototype:
    herr_t H5Gclose(hid_t group_id);

Purpose:
    Close a group ID from H5Gcreate() or H5Gopen()

API Change:
    <none>

Behavior Change:
    <none>

--------------------------------------------------------------------------------

Current Name:
    H5Gcreate

Prototype:
    hid_t H5Gcreate(hid_t loc_id, const char *name, size_t size_hint);

Purpose:
    Create new group

API Change:
    Will need to drop 'size_hint' parameter and acquire "group creation" and
    "group access" property list ID parameters.

Behavior Change:
    Property list settings will determine behavior changes.

--------------------------------------------------------------------------------

Current Name:
    H5Gget_comment

Prototype:
    int H5Gget_comment(hid_t loc_id, const char *name, size_t bufsize,
        char *buf);

Purpose:
    Get the "comment" on a group.

API Change:
    <none>

Behavior Change:
    <none> (deprecated)

--------------------------------------------------------------------------------

Current Name:
    H5Gget_linkval

Prototype:
    herr_t H5Gget_linkval(hid_t loc_id, const char *name, size_t size,
                              char *buf/*out*/);

Purpose:
    Retrieves the "link value" for a soft link in a group

API Change:
    <none>

Behavior Change:
    Will need to retrieve the "link value" of new "external" link type

    Possibly add H5Gget_linkval_by_idx routine?

--------------------------------------------------------------------------------

Current Name:
    H5Gget_num_objs

Prototype:
    herr_t H5Gget_num_objs(hid_t loc_id, hsize_t *num_objs);

Purpose:
    Retrieve number of objects in a group

API Change:
    <none>

Behavior Change:
    <none>

--------------------------------------------------------------------------------

Current Name:
    H5Gget_objinfo

Prototype:
    herr_t H5Gget_objinfo(hid_t loc_id, const char *name,
                          hbool_t follow_link, H5G_stat_t *statbuf/*out*/);

Purpose:
    Retrieve information about a link in a group

API Change:
    Update H5G_stat_t with any new information. (Including creation time, etc)

Behavior Change:
    Possibly add H5Gget_objinfo_by_idx routine?  (Absorb H5Gget_<foo>_by_idx
    routines, if so?)

--------------------------------------------------------------------------------

Current Name:
    H5Gget_objname_by_idx

Prototype:
    ssize_t H5Gget_objname_by_idx(hid_t loc_id, hsize_t idx, char* name,
        size_t size);

Purpose:
    Retrieve name of n'th object in a group.

API Change:
    <none>
    
Behavior Change:
    Choice of which index to use (and direction of use) will be determined by
    group access property list used in group create or open call.

--------------------------------------------------------------------------------

Current Name:
    H5Gget_objtype_by_idx

Prototype:
    H5G_obj_t H5Gget_objtype_by_idx(hid_t loc_id, hsize_t idx);

Purpose:
    Retrieve type of n'th object in a group.

API Change:
    <none>
    
Behavior Change:
    Choice of which index to use (and direction of use) will be determined by
    group access property list used in group create or open call.

--------------------------------------------------------------------------------

Current Name:
    H5Giterate

Prototype:
    herr_t H5Giterate(hid_t loc_id, const char *name, int *idx,
                          H5G_iterate_t op, void *op_data);

Purpose:
    Iterate through objects in a group, with user callback routine.

API Change:
    Change "int *idx" to "hsize_t *idx".

    Change to iterating over the objects in the "loc_id" instead of the
    objects in the object pointed to by loc_id/name (and drop the "name"
    parameter).

Behavior Change:
    Choice of which index to use (and direction of use) will be determined by
    group access property list used in group create or open call.

    Possibly allow recursive iteration through groups within current group?
    (Which would require an API change)

    Possibly allow use of regex for determining which objects to iterate over?
    (Maybe with a new API routine?)

--------------------------------------------------------------------------------

Current Name:
    H5Glink/H5Glink2

Prototype:
    herr_t H5Glink(hid_t src_loc, const char *cur_name, H5G_link_t type,
                        const char *new_name);
    herr_t H5Glink2(hid_t src_loc, const char *src_name, H5G_link_t type,
                        hid_t dst_loc, const char *dst_name);

Purpose:
    Creates a link to an object in another group, using names (as opposed to
    another type of index).

API Change:
    H5G_link_t will need to be expanded to include new "external" link type.

    We could add a property list to this routine to allow future link
    information (access control information, etc.) to be set for the new link.

Behavior Change:
    This routine will need to create the new "external" link type, which allows
    links to point to objects in other HDF5 files.

    The new link will automatically be assigned a creation time and inserted
    into the creation time index (if a creation time index is active for the
    group) when the new link is created.

--------------------------------------------------------------------------------

Current Name:
    H5Gmove/H5Gmove2

Prototype:
    herr_t H5Gmove(hid_t src_loc, const char *src, const char *dst);
    herr_t H5Gmove2(hid_t src_loc, const char *src, hid_t dst_loc, 
                        const char *dst);

Purpose:
    Renames a link to an object in current group, using names (as opposed to
    another type of index).

API Change:
    <none>

    We could add a property list to this routine to control how other link
    information (creation time and other link information/behavior we haven't
    thought of yet) will be set/modified for this link as it is moved.

Behavior Change:
    When a link is moved (ie. renamed) within a group, it's creation time is
    _not_ changed (and thus it's offset in the creation time index for that
    group is the same).  If the link is moved to another group, the link's
    creation time "counter" field may be modifed if existing object(s) in the
    destination group have duplicate creation second/nanosecond values.  The
    counter field of the new link will be modified in that case to be a value
    one greater than the highest counter for any object with that
    second/nanosecond.  The link will be inserted into the destination group's
    creation time index (if a creation time index is active for that group).

    Generally speaking, moving a link within a group or to another group should
    not modify any other aspect of the link information aside from the name of
    the link.

--------------------------------------------------------------------------------

Current Name:
    H5Gopen

Prototype:
    hid_t H5Gopen(hid_t loc_id, const char *name);

Purpose:
    Open an existing group

API Change:
    Add "group access" property list parameter

Behavior Change:
    Property list settings will determine behavior changes.

--------------------------------------------------------------------------------

Current Name:
    H5Gset_comment

Prototype:
    herr_t H5Gset_comment(hid_t loc_id, const char *name, const char *comment);

Purpose:
    Set a "comment" on a group.

API Change:
    <none>

Behavior Change:
    <none> (deprecated)

--------------------------------------------------------------------------------

Current Name:
    H5Gunlink

Prototype:
    herr_t H5Gunlink(hid_t loc_id, const char *name);

Purpose:
    Removes a link from a group to an object

API Change:
    <none>
    
Behavior Change:
    <none>
    
--------------------------------------------------------------------------------


==================
Added API routines:
==================

--------------------------------------------------------------------------------

Current Name:
    H5Gset_create_time

Prototype:
    herr_t H5Gset_create_time(hid_t loc_id, const char *name,
        int64_t new_seconds, uint32_t new_nanoseconds);

Purpose:
    Change the creation time of a link in a group (which may cause its location
    in a creation time index to change).

    If a creation time second/nanosecond is chosen which is already used by
    another link in the group, the counter field is used to internally
    disambiguate the two links.

    This routine is analogous to H5Gmove2, only operating on the creation time
    index instead of the name index.

--------------------------------------------------------------------------------

Current Name:
    H5Glink_by_idx

Prototype:
    herr_t H5Glink_by_idx(hid_t src_loc, const char *src_name, H5G_link_t type,
                        hid_t dst_loc, hsize_t dst_idx);

Purpose:
    Create a link to an object in another group by index order.

    Is this routine useful?

--------------------------------------------------------------------------------

Current Name:
    H5Gunlink_by_idx

Prototype:
    herr_t H5Gunlink(hid_t loc_id, hsize_t idx);

Purpose:
    Remove the n'th object (by current index order) in a group.

--------------------------------------------------------------------------------

Current Name:
    H5Gopen_by_idx

Prototype:
    herr_t H5Gopen_by_idx(hid_t loc_id, hsize_t idx);

Purpose:
    Open the n'th object in a group, according to the order with an index.

    Or, should this be H5Dopen_by_idx, etc?  (Which is how our APIs currently
    work)

--------------------------------------------------------------------------------

Current Name:
    H5Gget_num_indices

Prototype:
    herr_t H5Gget_num_indices(hid_t loc_id, unsigned *num_idx);

Purpose:
    Determine the number of indices available on links of a group

--------------------------------------------------------------------------------

Current Name:
    H5Gget_index_type

Prototype:
    herr_t H5Gget_index_type(hid_t loc_id, unsigned idx, H5G_index_t *type);

Purpose:
    Determine the type of the n'th index on links of a group

--------------------------------------------------------------------------------


New property lists:
==================

Name:                   Purpose:
=====                   =======
"Group Creation"        Control various parameters of a group, like it's B-tree
                        properties and whether it has an index allowing it
                        track the order that objects are created.  Possibly
                        other parameters as well.

"Group Access"          Control various parameters of accessing a group, like
                        the default index to use for various operations, etc.


Interesting ideas:
=================

- Make routines that operate on key of index more generic?
    - Accept name if index is on names, accept creation time if index is on
        creation time, etc.  We would do this by accepting a "void *" instead
        of a "char *" and interpreting the value pointed to as a string or a
        time, based on the index chosen for that group ID.
    - Affected routines:
        - H5Gget_linkval
        - H5Gget_objinfo
        - H5Giterate (?)
        - H5Glink/H5Glink2 (?)
        - H5Gmove/H5Gmove2 (?)
        - H5Gopen
        - H5Gunlink

- Allow which group is the "root" group to be changed