NOTE: The galeon mailing list is no longer active. The list archives are made available for historical reasons.
Hi Ben et al., Yes, I take your point here. Thanks also to John C for a very instructive post. I've learned a lot from this discussion and I'm aware that I could keep bugging everyone ad infinitum, so I'm going to have a go at summarizing some issues in a short essay (for my own benefit as much as anything else). Please let me know if I've got the wrong end of the stick at any point. Here goes (starting at the very beginning): The metocean community is standardizing around CF-NetCDF as a self-describing data format. Within the community such data are typically shared either as flat files (through HTTP/FTP etc) or through OPeNDAP servers, which provide aggregation and subsetting capabilities. A large amount of tooling has already been generated around this approach. The WCS specification has the potential to bring new capabilities, primarily: (1) the sharing of data and metadata with external communities who are familiar with WCS but not with OPeNDAP etc; and (2) the ability for all users to access data subsets based upon coordinate values rather than array indices. Other potential future benefits of WCS include the ability to use GetCoverage to access a _reference_ to a data (sub)set, rather than a file. This reference could be specified as an OPeNDAP URL or as a URL to a pre-extracted file on an HTTP server. This allows for "asynchronous" access in the case of large, time-consuming data extraction jobs (see the "WCSplus" effort), and also allows the reference to be passed to a downstream service in a workflow scenario. Data consumers that might wish to access a WCS server can be roughly divided into two categories. The first category of user (typically a metocean scientist) requires data to be transmitted without loss of information: i.e. no regridding or interpolation and probably no subsampling either. Such a user would need to specify data subsets either in the native CRS of the data (which might be difficult for curvilinear coordinate systems) or in index space (which replicates OPeNDAP functionality). These access patterns may require a high degree of sophistication on the part of the user and/or the client tools in question. Data would be delivered either as a CF-NetCDF file or an OPeNDAP URL. The second category of user requires a simple and familiar means of accessing data but does not care if the data are manipulated behind the scenes. Such users can be helped by providing access to metocean data in familiar projections such as lat-lon, and in familiar file formats such as GeoTIFF (for 2D slices only). (I think it would be valuable to expand the scope to include requests in other common CRSs, particularly polar projections - I tend to disagree with John C. that this would take us into the realm of WMS.) If these users require 3- or 4-D data subsets (which I anticipate is a minority of this class of users) they will have to download data in CF-NetCDF - some GIS tools are building in support for CF-NetCDF but it is probably unlikely that they will be able to interpret the CF conventions in their entirety for the foreseeable future. Work still remains to be done to pin down the core use cases in order to guide the community's limited effort and resources to the benefit of the maximum number of users. (It seems to me that aiming initially at "non-scientific" users will give a better return on investment than aiming at the scientists because we can offer more new capability - for the scientists we are essentially offering a new way of doing things they can do already, plus or minus a few deltas. This is only my view though and I do appreciate there is value in targeting both categories of user.) One question to finish - do the two (artificial) categories of user above map onto the WCS1.2 "core plus extensions" concept? I could imagine that the "non expert" users might use "core" WCS whereas the experts would use the extensions, but I'm almost completely ignorant of WCS1.2. Cheers, Jon On Mon, Sep 29, 2008 at 5:21 PM, Ben Domenico <Ben@xxxxxxxxxxxxxxxx> wrote:
Hi Jon and others, In response to the suggestion that users in the traditional GIS community can't make use of CF-netCDF files, there exists a fairly simple WCS use case that has worked for a couple years now in getting data in a useful CF-netCDF form to traditional GIS users -- most notably users of arcGIS in the hydrology and human impacts disciplines. In this case, the WCS client module is a very simple one that knows virtually nothing about CF-netCDF but does know how to use getCapabilities to learn what coverages are available, describeCoverage to request information about those coverages, and then getCoverage with a 3D bounding box, coverage name, and time specified in order to get back a CF-netCDF file that's stored on local disc. Since the release of arcGIS 9.2, the processing and display modules in arcGIS are able to read such CF-netCDF files and work with them. I've been working with several arcGIS users in academia and governement who have incorporated such a simple (python based) WCS client into arcGIS processing chains that make use of the sophisticated database operations and map display capabilities that GIS users are accustomed to. Please note that this is NOT an argument against geoTIFF output. Also there are many important use cases where it's important to have the WCS client read the coverage data directly into memory rather than into a file on local disk. And I am not arguing against an option that allows a client to request a OPeNDAP URL be returned by a getCoverage request. For certain applications, those are important options. But my point is that it has already been demonstrated that a large amount of useful work can be done with CF-netCDF datasets using traditional GIS systems by employing simple WCS client module that need not understand CF-netCDF itself but can gather information and request subset coverages in the form of a CF-netCDF file. The practicality and usefulness of the latter approach has already been proven by real users. Moreover Dominic Lowe's addition of a WCS client library to the open source OWSLib has made it even more straightforward to implement. So let's make sure we keep this option in the mix. -- Ben
galeon archives: