WCS-THREDDS Gateway

Ben Domenico
Draft last modified: January 16, 2005

THREDDS-Enabled Data Services Supported at Present Time

In the Unidata community framework of client/server data and metadata access systems, there are a number of client/server protocols available at different data provider sites. At the other end, some client applications can access data via some of the protocols while others can only access data via other protocols. THREDDS catalogs provide information about which datasets are avaiable via with services/protocols. The three main client/server (as opposed to full-file transfer with FTP or GridFTP) protocols for remote data access in use in the community are OPeNDAP, ADDE, and netCDF access via HTTP protocol. In many cases the data access systems are augmented and integrated with THREDDS catalog services which proved inventory list and metadata access. Thus client applications can learn what's available on the site via the THREDDS interface, then access the datasets themselves via OPeNDAP, ADDE, or netCDF/HTTP protocols.

OGC Data Services

Open Geospatial Consortium Web Mapping Servers (WMS) provide access to datasets that have been converted into visual map form. However Web Feature Servers (WFS) and Web Coverage Servers (WCS) enable access to datasets themselves. An oversimplified characterization of theses services is that WFS is used for traditional GIS vector (point, line, polygon) data whereas WCS is for coverage (image, grid) data.

The formats currently supported by WCS are:

• GeoTIFF, TIFF based interchange format for georeferenced raster imagery
• HDF-EOS, Heirarchical Data Format for the Earth Observing System
• DTED, Digital Terrain Elevation Data
• NITF, National Imagery Transmission Format
• GML, Geography Markup Language

OGC Augmented with NetCDF Dataset Interface

The goal is to add OWS (OGC Web Services) data access to THREDDS as a way to bridge between the GIS community and Unidata's geoscience community . We propose to add the NetCDF file format to the suite of WCS supported data formats.

WCS Gateway to THREDDS-Integrated Services

A very rough draft document outlining the salient issues related to this objective is NcML/NetCDF Dataset
as Form of OGC Web Coverage Service. The diagram below is an attempt to capture the key components of the WCS protocol interface layer we propose to implement on top of components that already are in operation at scores of THREDDS-integrated data provider sites.

As the figure shows, the gateway converses with the client via WCS protocols (getCapabilities, describeCoverage, and getCoverage). These requests are translated into the equivalent requests in using the interfaces and protocols of the THREDDS-integrated server:

• THREDDS inventory catalogs are accessed and used to generate dataset lists for the getCapabilities response
• NcML-GML metadata from the THREDDS server are used to construct the describeCoverage response. Note that there may be two initial prototype implementations: a simpler one that just provides NcML with Coordinate System extensions and a second that provides theNcML-GML dialect.
• netCDF objects are retrieved from the netCDF/OPeNDAP server either directly via the netCDF interface or from an OPeNDAP service.
• The netCDF objects are then transformed into one of three forms (geoTIFF, GML, or netCDF) for transmission back to the client in response to the getCoverage request.

For a detailed description of NcML-GML, please see NetCDF Markup Language (NcML) and its GML-based extension (NcML-GML).

WCS-THREDDS server specification and implementation summary

• WCS-THREDDS server: a gateway between UNIDATA THREDDS and OGC-WCS
• WCS-THREDDS implements:
  • THREDDS client functionalities;
  • WCS extended operations
  • ncML-GML engine
• WCS-THREDDS mediates between the THREDDS model and the OGC-WCS model, resorting to ncML-G model

Broad Experiment Objectives

The objectives for specific technical components of the experiment are given in sections below, but the overall high level objectives are to:

• Work with existing WCS client applications to see whether they can make use of the experimental WCS server
• Compare three options for WCS data encoding:
  • geoTIFF
  • NcML-GML
  • netCDF
• Provide a WCS prototype that allows traditional GML client applications to access data inthe NcML-GML encoding.
• Provide a WCS prototype that allows traditional netCDF client applications to access data via the WCS protocol.

How Do We Get There?

Initial Steps

• THREDDS Dataset Inventory Catalogue extension
• WCS specification extension
• WCS-THREDDS server specification and implementation
• THREDDS and WCS Interoperability Experiment
  • Use cases
  • Rationale
  • Architecture
  • Data Model Mediation

THREDDS Dataset Inventory Catalogue extension

Experiment objective for getDatasetInventoryCatalogue operation:

• to determine whether THREDDS model completely supports the ncML-G model
• To specify extension to the THREDDS model (e.g. to accommodate GIS specific metadata) if needed
• To specify a new netCDF convention if needed

WCS specification extension

Experiment objectives for getCapabilities operation:

• To determine whether OGC-WCS model completely supports the ncML-G model
• To specify extension to the OGC-WCS model (e.g. exploiting the metadataLink element)

Experiment objectives for describeCoverage operation:

• To determine whether OGC-WCS model completely supports the ncML-G model
• To specify extension to the OGC-WCS model (e.g. exploiting the metadataLink element)

Experiment objectives for getCoverage operation:

• To extend OGC-WCS model to support
  • netCDF file format
  • ncML-GML dialect of GML
    
    

THREDDS and WCS getCapability Actions

Client request: WCS client accesses a WCS-THREDDS server issuing a getCapabilityrequest

Server actions:

• The WCS-THREDDS server accesses a THREDDS service and issues a getDatasetInventoryCatalogue request
• THREDDS service returns a DatasetInventoryCatalogue document -extended to support the ncML-G data model
• The WCS-THREDDS server returns anCapabilities document -extended to support the ncML-G data model

THREDDS and WCS describeCoverage Actions

Client request: WCS client accesses the WCS-THREDDS server issuing describeCoverage request

Server Action: The WCS-THREDDS server returns a CoverageDescription document -extended to support the NcML-G data model. (Note again that one prototype may only support NcML with Coordinate System extensions.)

THREDDS and WCS geoTIFF Coverage Use Case

Objective: To access a netCDF dataset through WCS-THREDDS server; to serve the dataset as GeoTIFF file

Client request: WCS client accesses the WCS-THREDDS server issuing a getCoverage request, asking for the following output format: GeoTIFF

Server action: WCS-THREDDS server returns the requested Coverage as a GeoTIFF file.

THREDDS and WCS NcML-GML Coverage Use Case

Objective: To access a netCDF dataset through WCS-THREDDS server; to serve the dataset as NcML-GML document

Client request: WCS client accesses the WCS-THREDDS server issuing a getCoverage request, asking for the following output format: GML

Server action: WCS-THREDDS server returns the requested Coverage as a GML document

THREDDS and WCS netCDF Coverage Use Case

Objective: To access a netCDF dataset through WCS-THREDDS server; to serve the dataset as netCDF file

Client request: WCS client accesses the WCS-THREDDS server issuing a getCoverage request, asking for the following output format: netCDF

Server action: WCS-THREDDS server returns the requested Coverage as a binary netCDF file.

Datasets and Catalogs for Prototype System

For purposes of the initial demonstration of the prototype system, we will use netCDF datasets generated from the output of numberical weather forecast model runs. These datasete are typical of those used in the atmospheric sciences in that they have multiple dependent variables that vary in three spatial dimensions and time. In fact they have multiple temporal dimensions in that there is the time of the model run and the relative forecast time.

A catalog of forecast model output covering a summer rain event in Colorado have been selected for the initial prototype demonstration. The THREDDS catalog is at:

http://my.unidata.ucar.edu/content/projects/THREDDS/DataPublications/Case2004Jul16.xml

Note that this catalog points not only to the datasets in the case study but also to the "latest" model run as well.

These datasets are describe in

http://my.unidata.ucar.edu/content/projects/THREDDS/DataPublications/SampleDataPublicationFor2005AMS.htm

On a system with 500MB of memory and Java WebStart installed, one can actually bring up the IDV (Integrated Data Viewer) which in turn will access the datasets from the server and allow the reader to interact directly with them.( The IDV is an exising client that can access many datasets on the current set of THREDDS-integrated data servers, but it doesn't implement the WCS interface as yet. It would be interesting to find WCS client experts who might want to work on creating a WCS-enabled version of the IDV.