[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 19990304: Future work on netCDF

>From: Paul van Delst <address@hidden>
>Subject: Future work on netCDF
>Organization: Space Science and Engineering Center
>Keywords: 199903041803.LAA10052 netCDF

Paul,

> Thank you very much for the netCDF references. While reading that page I
> came across the "what future work is planned for netCDF". I would like
> to put my two cents worth in on some of the items in the list (I realise
> this probably isn't the forum), those being:
> 
>       Fortran-90 interface  <--- ** eagerly awaited **
>       Multiple unlimited dimensions per dataset
>       Support for nested arrays (as in J), ragged arrays, and
> structures.
>       Support for new types: 64-bit integers, unsigned types
> 
> Some research institutions in the satellite remote sensing community are
> beginning to make a change from requiring code to be written in
> Fortran-77 to it being written in Fortran-90. Right now we use
> Fortran-90 but with the F77 netCDF interface. Satellite data streams are
> full of different types of data types (due to bandwidth limitations) and
> unsigned type are one of those. Support for this capabililty would be
> welcomed.

You have reached the right forum, since I'm largely responsible for
deciding how to allocate the work of our small group of developers (6
software engineers).  Since that paper on plans for netCDF was
written, our priorities have changed significantly.  We're now working
to deliver platform-independent applications for research and
education in the atmospheric sciences by creating a component-oriented
framework that supports easy construction of custom applications,
while maintaining, supporting, and enhancing some existing software
application packages.

Our recent netCDF focus has been developing the Java version to
support the other platform-independent applications and framework.  We
don't have any Fortran-90 expertise, and have been trying to enlist
the aid of others to help to develop a Fortran-90 version.

Over a year ago, we discussed some ideas for a Fortran-90 interface
with Robert Pincus of NASA.  Pincus convinced us that it's possible to
use generic functions and optional (named) arguments to implement a
Fortran90 interface (purely with module definitions) that uses just
one read and one write interface, instead of the current netCDF 3.4
Fortran-77 version that uses 30 read interfaces and 30 write
interfaces.  In case you are interested, I've appended some email
about this subject ...

The other items in our netCDF wish list are likely to be implemented in
Java first, as the need for them arises in the context of our other
work.  But thanks very much for the feedback on priorities.  We may
change direction again, and it's important to keep user priorities in
mind, not just what programmers think is neat to work on this week.

--Russ

_____________________________________________________________________

Russ Rew                                         UCAR Unidata Program
address@hidden                     http://www.unidata.ucar.edu



    To: Robert Pincus <address@hidden>
            address@hidden
    Subject: Re: Fortran 90 support in netCDF 3.3.1 
    In-reply-to: Your message of "Thu, 30 Oct 1997 18:42:34 EST."
                 <v03010d0db07ebc671886@[128.183.200.25]> 
    Date: Fri, 31 Oct 1997 12:32:55 -0700
    From: Russ Rew <address@hidden>

    > To: Russ Rew <address@hidden>
    > From: Robert Pincus <address@hidden>
    > Subject: Fortran 90 support in netCDF 3.3.1

    Robert,

    > Wow, that Fortran 90 module for netCDF 3.3.1 looks like a lot of work.

    Only a couple of hours with emacs macros ...

    > I have been using another approach to get at the number of bytes used in
    > different representations. At
    > ftp://climate.gsfc.nasa.gov/pub/pincus/F90/typeSizes.f90 is a module I
    > wrote to try and get around the problem of specifying integer*1, etc. It
    > defines symbolic type parameters for one, two, and four byte integers, and
    > four and eight byte reals. There is also a Boolean function which would be
    > invoked during testing to ensure that the reported kind parameters are
    > consistent, since some compilers don't support all those different kinds.
    > Feel free to include whatever you need from the module in the netCDF
    > release.

    Thanks, this looks useful.

    > I think you may want to change the interface blocks to the Fortran 77
    > functions. On some compilers, if a variable is declared as assumed shape
    > (i.e. has the declaration dimension(:) rather than dimension(*)) the
    > compiler passes along extra information to let the called routine 
determine
    > the bounds and size of the array. Your F77 functions won't know what to do
    > with that, so I suspect all you should replace every occurrence of
    > dimension(:) with dimension(*).

    OK, I've done that.

    > I remember having a discussion with you about the interfaces last summer,
    > and I wondered how you were going to be able to match both type (as you
    > have in this model) and rank. In the module you sent me it appears that 
one
    > can only one dimension arrays. For example, if one tried to pass a two
    > dimensional array as the IVALS argument to
    > 
    > INTEGER FUNCTION NF_PUT_VARA_INT(NCID, VARID, START, COUNT,IVALS)
    >   ....
    >   integer,         dimension(*), intent ( in) :: IVALS
    > end function NF_PUT_VARA_INT
    > 
    > the compiler would complain that the argument had the wrong rank (number 
of
    > dimensions). The user could get around this problem by PACKing the values
    > from a multi-dimensional array before handing them off to the NF_PUT_VARA_
    > routines, but you would probably want to note that in the documentation.
    > (I'm pretty sure there is no way to build an interface block that tells 
the
    > compiler that the underlying routine doesn't care about the rank of the
    > array.)

    Oops, I forgot about that.  That makes the whole idea of adding extra
    checking with an interface block kind of useless.  I just tried
    inserting some "use netcdf" statements in various nf_test subroutines,
    and saw errors caused by this.  How do math libraries like IMSL that
    exploit the fact that you can use a vector or a matrix for the RHS of a
    linear system (for example) deal with this?  This seems like a mistake
    in the standard ...

    > I have a couple of suggestions as to ways you could exploit the neat
    > features of Fortran 90, but I don't know what level of resources and/or
    > expertise you have to devote to this. I will say that a lot of numerical
    > people use netCDF, and many of us are moving to Fortran 90. I would be
    > happy to help you with some of these ideas as my time allows.

    Currently, we have no expertise or resources to adapt netCDF to Fortran
    90.  We're trying to recruit someone from NCAR's Scientific Computing
    Division to help with that.

    > Have you given any thought to making generic versions of some functions?
    > For example, you could use the netCDF module to define a generic function
    > for putting one dimensional arrays. I put a free-standing module to do 
that
    > at ftp://climate.gsfc.nasa.gov/pub/pincus/F90/netcdfF90.f90. Using this
    > module, one could make a call to
    >   NF90_PUT_VARA(NCID, VARID, START, COUNT, VALUES)
    > where VALUES could be of any type (1, 2, 4 byte ints, single precision and
    > double precision reals, or text). I think this would be a nice addition.
    >
    > The idea of generic functions could be expanded to allow one to write
    > multi-dimensional arrays using the current interface. One would have to
    > write a LOT of specific functions: 7 (to include up to 7 dimensional 
arrays,
    > the max in the standard) times 6 (3 kinds of ints, 2 reals, 1 text) is 42.
    > But one could also do some error checking in the specific functions to
    > ensure that start, count, and stride were consistent with the array 
section
    > being passed, since Fortran 90 (unlike C or F77) knows how many values are
    > being passed. That could be really neat.
    > 
    > As icing on the cake, one could use the optional argument features of
    > Fortran 90. One obvious place is if one is writing every element of an
    > array. Since netCDF can check the file definition to see the shape of the
    > array in the file, one could make start, count, and stride optional
    > arguments, which only needed to be supplied if one were writing a
    > subsection. That would enable calls like
    >   NF90_PUT_VARA(NCID, VARID, VALUES)
    > which would also be cool.

    Yes, this looks very useful.  I'll save your message, in case we get
    resources for adapting netCDF to Fortran 90.

    Thanks for the information.

    --Russ

    _____________________________________________________________________

    Russ Rew                                         UCAR Unidata Program
    address@hidden                     http://www.unidata.ucar.edu