Unidata directions: At the crossroads?
By Robert Fox
Chair, Unidata Policy Committee Executive Director
Space Science and Engineering Center
University of Wisconsin-Madison
NOTE: This article first appeared in The
UCAR NewsletterVol. 14, No. 6, November-December 1990
Where's the Unidata Program going? Good question, and one that members of
the Unidata Policy Committee frequently hear. The best description of
current directions (and the best concise history of Unidata) are in the
three-year proposal to the NSF for continuing the Unidata Program (Unidata's
Next Steps, A Three Year Proposal; 13 November 1989; copy available upon
request from Unidata Program, P.O. Box 3000, Boulder, CO 80307-3000; e-mail:
sally@unidata.ucar.edu.). NSF has recently funded Unidata on the basis of
this proposal.
There are some long-range plans that are conceptual and subject to debate
and evolution. In preparing this article, I've drawn on a five-year planning
document. written for the UCAR trustees by Unidata director David Fulker,
that discusses these ideas.
The reasons for Unidata
The Unidata Program came into being to solve two problems perceived by the
academic atmospheric sciences community as major: university (I'll use the
word generically to mean any institution of higher education) access to and
acquisition of atmospheric science data, and access to interactive tools for
conducting education and research with those data.
The first problem arose when the National Weather Service (NWS) announced
it would discontinue various modes of communicating data to its field sites
in favor of the AFOS (Automation of Field Operations and Services)
program. The community had grown accustomed to piggybacking on the existing
modes for acquiring their conventional and facsimile data, and faced the
prospect of being unable to access these data under AFOS. (In the end, the
NWS continued several of the older communication modes or found
alternatives.)
The second problem came about because few universities in the atmospheric
sciences could afford to develop the interactive processing software and
satellite data acquisition systems they needed. NSF was receiving multiple
requests to develop such capabilities and desired a broader "community"
solution to this problem rather than funding the invention of many similar
wheels.
There were many other reasons for Unidata, such as data access to and from
field experiments, access to historical data, and interaction with
supercomputers, but I se the above as the main forcing functions.
Unidata to date
Unidata now supports about 90 sites and distributes a number of software
packages. The packages include Unidata McIDAS (a version of the University
of Wisconsin's Man-computer Interactive Data Access System, which uses a
special data stream to provide satellite and other weather data that may be
analyzed and displayed on an IBM PS/2 computer), the Local Data Manager
(LDM, which captures raw weather data), network Common Data Format (netCDF,
a software library with a standardized method of storing and retrieving
scientific data), WXP (analysis and display software developed by Purdue
University to run with the LDM and netCDF), the Unidata Campus Weather
Display (a prototype system for distributing weather information across a
network), and GUISE (a series of routines to help programmers develop new
display applications).
These packages entail a substantial and ongoing support effort: holding four
training workshops every year, consulting with sites having installation or
use problems, creating and distributing documentation, and maintaining
existing software in the face of changes in the underlying hardware and
operating systems.
Plans for the next couple of years, where the vast majority of the Unidata
funds and resources will be placed, are explicitly detailed in the Unidata
proposal to NSF. The longer term is more speculative but very intriguing. We
hope to extend Unidata capabilities through more powerful software and
access to historical data while continuing to provide essential support to
the growing body of university users. Some of our goals are:
- A. Software enhancement and support: to continue improving Unidata software,
to reduce the number of supported operating systems and software versions in
an evolutionary fashion, and to fully support those systems that have been
fielded as long as they are in use. Specifically, this includes adding a
UNIX-based image analysis package for displaying and analyzing digital
satellite images, adding the GUISE interface so programmers can more easily
add their own programs to the system, adding routines to independently
manipulate data stored in Unidata netCDF files, standardizing graphics used
in Unidata software, and distributing a UNIX version of NASA's GEMPAK
software package for analyzing and displaying meteorological data.
Unidata will continue to distribute and support the McIDAS package as
developed and furnished by the University of Wisconsin Madison, and to
augment the interoperability of McIDAS and the LDM, so that Unidata's two
software systems can share increasing amounts of data. Unidata will also
continue to augment its Campus Weather Display software, making it easier to
use for weather briefings and for displaying weather information across a
campus.
- B. Data access: to continue to facilitate university access to current
weather data and adapt Unidata systems to receive the NOAAport
high-bandwidth data. (NOAAport will be a data broadcast used by the Advanced
Weather Interactive Processing System, now being designed by the NWS.)
Unidata also plans to develop software "translators" that will convert data
in major data centers from their archival formats to the standard form
(netCDF) used in Unidata.
- C. Research and development: to undertake a very small amount of research
and development in the areas of standardized data access, applications
interfaces, and user interfaces, and to incorporate the results into
Unidata's LDM and netCDF software.
- D. Liaison to Unidata-related programs:
- COMET, STORM, and NOAA's Forecast Systems Laboratory: It is hoped that
coordinating with UCAR's Cooperative Program for Operational Meteorology,
Education, and Training; the National Stormscale Operational and Research
Meteorology Program; and the Forecast Systems Laboratory (which is
continuing to develop PROFS, the Program for Regional Observing and
Forecasting Services capabilities) will provide opportunities for new
university capabilities like access to COMET case studies, access to
experimental data sources such as wind profilers and Doppler radars, and
cooperative NOAAport data reception development. Similar benefits will
derive from cooperation with the STORM program.
- EOSDIS: NASA's satellite-based Earth Observing System (EOS) has a very
large component called EOSDIS (EOS Data and Information System) to manage
the data generated. One of the EOSDIS objectives is to facilitate university
access to EOS data, a goal similar to that of the Unidata Program. Close
coordination should yield benefits by modifying the evolutionary directions
of both programs, allowing each to profit from the experiences of the other.
- Education initiatives: A number of individuals and organizations are
seeking to use Unidata experiences and capabilities to enhance education in
a variety of ways, such as minority initiatives, data receipt at
pre-university institutions, and curriculum development UCAR has begun
exploring possibilities for programmatic synergism in these areas.
The longer term
Unidata began life with rather narrowly defined objectives for developing
and supporting data and technology capabilities for the university
atmospheric sciences community. A large number of these objectives have been
met, but many specific capabilities must still be implemented.
The Unidata Policy Committee, while continuing to focus on the specific
objectives defined in the original and subsequent proposals and in Unidata's
charter, increasingly finds itself discussing a much broader range of data
access and interactive processing matters. For example, what are appropriate
roles for Unidata in the EOS era? The latest Unidata proposal did not detail
significant effort in this arena, and the EOS concept did not exist when the
original Unidata mission was defined. Nonetheless, the similarities and
possible synergies between Unidata and the planned EOSDIS are unmistakable.
It seems to me that Unidata is becoming more and more a platform for
considering the general technological welfare of the community as it
pertains to campus receipt and processing of atmospheric sciences data. The
various Unidata committees are broadly representative of the entire
community, and the program has recently had several extensive reviews. The
university community is aware of, and involved in, this evolution that I se
occurring in the Unidata mission. I believe that Unidata's mission will
continue to broaden, perhaps even into the geophysical and ocean sciences.
This is an exciting time for Unidata and, over the next few years, the
program could move in a number of new directions. I encourage individuals
and organizations within the community to reflect upon and make known their
views of Unidata and its mission. Should Unidata stick to a narrow role in
the technology of the atmospheric sciences, seeking minimal coordination
with related organizations? Or should it increasingly represent the wider
university earth sciences community with broader technical capabilities and
systems? You tell us.
Matthew Hicks <matt@unidata.ucar.edu>
Last modified: Thu Dec 29 16:10:58 1994
