[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Adding a service to a remotely served dataset



Very interesting. Thanks John. Is this in the version we have (3.16.13) or is it in a new download?

When we get a chance, we will test this as it is a feature we would like to be able to add.

Regards,

-Roy

On Jul 9, 2007, at 9:32 AM, John Caron wrote:

We made an extension of the WCS Dataset URL format to allow the TDS to serve remote datasets. Identify the dataset by adding the parameter dataset whose value is a URL:

 http://servername:8080/thredds/wcs?dataset=datasetURL&;

The URL must be a dataset readable by the NetCDF-Java library, typically an OPeNDAP dataset on another server. It must have gridded data, with identifiable coordinate systems, etc. For example, an OPeNDAP URL might be

http://las.pfeg.noaa.gov/cgi-bin/nph-dods/data/oceanwatch/nrt/gac/ AG14day.nc
This can be served remotely as a WCS dataset with this URL:

http://servername:8080/thredds/wcs?dataset=http:// las.pfeg.noaa.gov/cgi-bin/nph-dods/data/oceanwatch/nrt/gac/AG14day.nc&


This is non-standard WCS; we havent emphasized this feature until we have a chance to review security implications. The WCS service now if off by default, and must explicitly be turned on in threddsConfig.xml, mostly because of this feature. We should probably seperately allow users to enable/disable "remote WCS access".


Oliver Newell wrote:
Hi Roy -
FYI - We're currently experimenting with implementing WCS services, and the same requirement (wish?) as what you're describing has been raised. Basically, the need to discover a data set via a cataloging mechanism (THREDDS, an OGC catalog, a DOD NCES catalog) and then be able to access it via a WCS that resides somewhere on the network. My initial take on it is that since WCS data access requests work with unique identifiers (not URLs), this limits the capabilities of the service in the sense that the Id-to- resource (data file or db) mapping is managed locally by the WCS. In other words, the WCS has to 'know about' the resource in advance of the request, in order to do the mapping. I think the WCS service should support a mode of operation where a resource's URL is included in the requests to the service, in order to support the external WCS concept. I think it is still important to maintain the Id as a separate entity, so it is not just a question of using a URL for the coverage identifier, though that idea might be tempting. The idea needs flushing out a bit, but it is still early enough in WCS's development that changes like this can be incorporated into the spec via the OGC working group (my opinion). It definitely seems like it would be a useful capability to address the situation you describe.
-Oliver
Roy Mendelssohn wrote:
Hi Glenn:

No problem. But we were interested in the general problem, as there are other datasets which also do not have WCS and which we would like to add it, rather than trying to get each of them to add it (Some just have an OPeNDAP server, not a THREDDS server). Yours allowed me to give Ethan a concrete example so he could look at the catalogs.

Thanks,

-Roy
On Jul 9, 2007, at 7:52 AM, Glenn.Rutledge wrote:

Hello Roy-
That particular dataset will be turned on shortly via WCS. That was an oversight. Nice to have users telling us where we need stuff! Glenn

Ethan Davis wrote the following on 7/6/2007 6:02 PM:
Hi Roy,

Looks like you are referencing NCDC catalogs rather than serving the datasets yourself. So, users would get redirected to the NCDC server and access the data through the NCDC server. So, as things stand, NCDC would have to add the WCS service.

You could point to each NCDC dataset with an NcML wrapper and actually serve the data from your server rather than redirecting to NCDC. If you did that, you could add the WCS service. However, the performance wouldn't be as good as direct from NCDC.

I'm afraid I don't know about the WCS request for a time range. So, I'll leave that for others to answer.

Ethan

Roy Mendelssohn wrote:
Hi Ethan:

I am not certain I follow.  To be more specific, look at:

http://oceanwatch.pfeg.noaa.gov/thredds/catalog.html

and scroll down to the datasets " Remote test to Nomads THREDDS catalog", which is actually on a THREDDS server on an NCDC computer. They are remotely served through catalogs. Can we add the WCS service on our end, or will it only work if NCDC adds the WCS service.

While I have your attention, I have not been able to get a straight answer if there is a way to send the WCS request to get a time range, rather than a single time, and if so what is the syntax for that.

Thanks,

-Roy

On Jul 6, 2007, at 11:24 AM, Ethan Davis wrote:

Hi Roy,

If you are actually serving the dataset through your server (which it sounds like you are since you mention existing NcML for the dataset) rather than providing a link in your catalog to the remote server, I believe you can simply add the WCS service to your TDS configuration catalog and serve the data via WCS. It shouldn't require any changes to the NcML, just to the service element in the catalog.

Of course, this depends on the dataset meeting the WCS server requirements, e.g., evenly spaced grid.

I'm not sure how it would perform. But I'll leave it for John to respond on that.

Ethan

Roy Mendelssohn wrote:
Hi:

The question for today is if our TDS is "serving" a dataset remotely (ie. the remote site has their own TDS and we just link indirectly to it), can we add a service to our version that they don't have (just like we can aggregate a remote dataset that is not aggregated). In particular the remote dataset only has an OPeNDAP response, but we would like to be able to respond to both an OPeNDAP and a WCS response.

If we change our NcML appropriately, will that work?

Thanks in advance,

-Roy

**********************
"The contents of this message do not reflect any position of the U.S. Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail: address@hidden (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."



=============================================================== ================
To unsubscribe thredds, visit:
http://www.unidata.ucar.edu/mailing-list-delete-form.html
=============================================================== ================

--Ethan R. Davis Telephone: (303) 497-8155 Software Engineer Fax: (303) 497-8690 UCAR Unidata Program Center E-mail: address@hidden
P.O. Box 3000
Boulder, CO 80307-3000 http:// www.unidata.ucar.edu/ ---------------------------------------------------------------- -----------



**********************
"The contents of this message do not reflect any position of the U.S. Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail: address@hidden (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."




--Glenn K. Rutledge
Services Team Leader
Remote Sensing and Applications Division
NOMADS Project Manager
National Oceanic and Atmospheric Administration
National Climatic Data Center
Asheville NC 28801
Phone: (828) 271-4097
Fax: (828) 271-4328

NOMADS: http://nomads.ncdc.noaa.gov/


**********************
"The contents of this message do not reflect any position of the U.S. Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division   Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail: address@hidden (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."



==================================================================== ===========
To unsubscribe thredds, visit:
http://www.unidata.ucar.edu/mailing-list-delete-form.html
==================================================================== ===========
===================================================================== ========== To unsubscribe thredds, visit:
http://www.unidata.ucar.edu/mailing-list-delete-form.html
===================================================================== ==========

**********************
"The contents of this message do not reflect any position of the U.S. Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division 
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail: address@hidden (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."



===============================================================================
To unsubscribe thredds, visit:
http://www.unidata.ucar.edu/mailing-list-delete-form.html
===============================================================================