Re: Other 'standard' formats

• To: mchance@xxxxxxxxxxxxxxx (Morrell Chance)
• Subject: Re: Other 'standard' formats
• From: "Len Makin, Supercomputing Support Group, CSIRO DIT Melbourne." <len@xxxxxxxxxxxxxxxx>
• Date: Wed, 19 Feb 1992 16:38:59 +1100

Among others available publically with interfaces of course is CDF - similar
data model as netCDF (obviously) but different APIs, physical storage (it
supports several interchangeable forms), and performance characteristics.
Code, utilities and docs. for Unix on anon ftp via ncgl.gsfc.nasa.gov and for
VMS via nssdca.gsfc.nasa.gov.  Platforms include SunOS, VMS, Ultrix, AIX, and
IRIX with MacOS, DOS and Unicos shortly (maybe HP-UX as well).

FITS, the std. interchange for astronomical imagery and related data (except
planetary) is well defined and used, but no API.  Hence, portablility
(ignoring big-endian/little-endian) is not a problem if you don't
mind grovelling in the bytes.  Documentation on FITS is also available
through nssdca.gsfc.nasa.gov.

Our group (Scientific Visualization Systems at TJ Watson Research Center) has
developed yet another format, which is based upon a more comprehensive data
model (cf, Haber et al, Proc. IEEE Visualization'91) that includes curvilinear
and irregular meshes and hierarchies, among other things.  Currently, the
physical disk-based format is complete and has been published, but it is
simple (e.g., sequential access).  A portable API for it is not yet
available, and hence support for disk-based structures and data base-type
updates in place are not supported.  But it's new (i.e., only in the last
couple of months).

The SIGGRAPH report was intended as an abstract for a 3-foot stack of documents
that I have as well as the deliberations at our two-day workshop.  FYI, Russ
Rew (of Unidata) and people associated with CDF, HDF, etc. were participants.
I hope to do something more comprehensive some time in the hopefully, not
too distant future.

If I can briefly answer some questions on some of the reported formats and
interfaces, please let me know.  I would also like to know more about the
formats that you cite.

Thanks,

Lloyd Treinish
------------------------------------- Referenced Note -------------------------
Received: from unidata.ucar.edu by watson.ibm.com (IBM VM SMTP V2R2) with TCP;
   Wed, 19 Feb 1992 00:54:01 EST
Received: by unidata.ucar.edu id AA29544
  (5.65c/IDA-1.4.4 for netcdfgroup-send); Tue, 18 Feb 1992 22:36:58 -0700
Received: from shark.mel.dit.CSIRO.AU by unidata.ucar.edu with SMTP id AA29540
  (5.65c/IDA-1.4.4 for <netcdfgroup@xxxxxxxxxxxxxxxx>); Tue, 18 Feb 1992 
22:36:53 -0700
Received: from trout.mel.dit.CSIRO.AU by shark.mel.dit.csiro.au with SMTP id 
AA26328
  (5.65c/IDA-1.4.4/DIT-1.3 for netcdfgroup@xxxxxxxxxxxxxxxx); Wed, 19 Feb 1992 
16:36:48 +1100
Message-Id: <199202190536.AA26328@xxxxxxxxxxxxxxxxxxxxxx>
Cc: netcdfgroup@xxxxxxxxxxxxxxxx, len@xxxxxxxxxxxxxxxx
             <9202190453.AA14363@xxxxxxxxxxxxxxx>

>
> The netCDF file system seems quite elaborate, with excellent features,
> and seems to have a popular following. As someone new to these so
> called standards, I am not quite sure which to choose.  There seem to
> be other interfaces around which have some claims to being 'standard'
> also.  Like, for example, the NCSA HDF system. And what about others
> like WEB and BASIS? There may also be others. Has anyone compared
> these against each other? Is there any sense in such a comparison?

A reasonably comprehensive yet brief summary was published as
"SIGGRAPH '90 Workshop Report - Data Structures and Access Software
for Scientific Visualization" in Computer Graphics Vol.25 Number 2 April 1991
a publication of ACM SIGGRAPH. The Workshop Chairperson/Organizer/Editor
was Lloyd Treinish  (lloyd@xxxxxxx) . HDF and netCDF are compared in
SCD Computing News February 1991 "Advantages and Disadvantages of HDF and
netCDF" G.Davis & R.Rew from UCAR.

| Len Makin,  CSIRO, Division of Information Technology (Melbourne)    |
|             Supercomputing Support Group                             |
|723 Swanston St, Carlton| (TEL) +61 3 282 2622   (FAX) +61 3 282 2600 |
|VIC 3053, Australia     | Internet:len@xxxxxxxxxxxxxxxx               |