Adding a service to a remotely served dataset
John Caron
caron at unidata.ucar.edu
Mon Jul 9 11:36:44 MDT 2007
Its been there for a while. Ill probably add enable/disable "remote WCS access" to next version, so we dont have to worry about security unless you have it turned on.
Roy Mendelssohn wrote:
> Very interesting. Thanks John. Is this in the version we have
> (3.16.13) or is it in a new download?
>
> When we get a chance, we will test this as it is a feature we would like
> to be able to add.
>
> Regards,
>
> -Roy
>
> On Jul 9, 2007, at 9:32 AM, John Caron wrote:
>
>> We made an extension of the WCS Dataset URL format to allow the TDS to
>> serve remote datasets. Identify the dataset by adding the parameter
>> dataset whose value is a URL:
>>
>> http://servername:8080/thredds/wcs?dataset=datasetURL&
>>
>> The URL must be a dataset readable by the NetCDF-Java library,
>> typically an OPeNDAP dataset on another server. It must have gridded
>> data, with identifiable coordinate systems, etc. For example, an
>> OPeNDAP URL might be
>>
>> http://las.pfeg.noaa.gov/cgi-bin/nph-dods/data/oceanwatch/nrt/gac/AG14day.nc
>>
>> This can be served remotely as a WCS dataset with this URL:
>>
>> http://servername:8080/thredds/wcs?dataset=http://las.pfeg.noaa.gov/cgi-bin/nph-dods/data/oceanwatch/nrt/gac/AG14day.nc&
>>
>>
>>
>> This is non-standard WCS; we havent emphasized this feature until we
>> have a chance to review security implications. The WCS service now if
>> off by default, and must explicitly be turned on in threddsConfig.xml,
>> mostly because of this feature. We should probably seperately allow
>> users to enable/disable "remote WCS access".
>>
>>
>> Oliver Newell wrote:
>>> Hi Roy -
>>> FYI - We're currently experimenting with implementing WCS services,
>>> and the same requirement (wish?) as what you're describing has been
>>> raised. Basically, the need to discover a data set via a cataloging
>>> mechanism (THREDDS, an OGC catalog, a DOD NCES catalog) and then be
>>> able to access it via a WCS that resides somewhere on the network. My
>>> initial take on it is that since WCS data access requests work with
>>> unique identifiers (not URLs), this limits the capabilities of the
>>> service in the sense that the Id-to-resource (data file or db)
>>> mapping is managed locally by the WCS. In other words, the WCS has to
>>> 'know about' the resource in advance of the request, in order to do
>>> the mapping.
>>> I think the WCS service should support a mode of operation where a
>>> resource's URL is included in the requests to the service, in order
>>> to support the external WCS concept. I think it is still important
>>> to maintain the Id as a separate entity, so it is not just a question
>>> of using a URL for the coverage identifier, though that idea might be
>>> tempting.
>>> The idea needs flushing out a bit, but it is still early enough in
>>> WCS's development that changes like this can be incorporated into the
>>> spec via the OGC working group (my opinion). It definitely seems like
>>> it would be a useful capability to address the situation you describe.
>>> -Oliver
>>> Roy Mendelssohn wrote:
>>>> Hi Glenn:
>>>>
>>>> No problem. But we were interested in the general problem, as there
>>>> are other datasets which also do not have WCS and which we would
>>>> like to add it, rather than trying to get each of them to add it
>>>> (Some just have an OPeNDAP server, not a THREDDS server). Yours
>>>> allowed me to give Ethan a concrete example so he could look at the
>>>> catalogs.
>>>>
>>>> Thanks,
>>>>
>>>> -Roy
>>>> On Jul 9, 2007, at 7:52 AM, Glenn.Rutledge wrote:
>>>>
>>>>> Hello Roy-
>>>>> That particular dataset will be turned on shortly via WCS. That
>>>>> was an oversight. Nice to have users telling us where we need
>>>>> stuff! Glenn
>>>>>
>>>>> Ethan Davis wrote the following on 7/6/2007 6:02 PM:
>>>>>> Hi Roy,
>>>>>>
>>>>>> Looks like you are referencing NCDC catalogs rather than serving
>>>>>> the datasets yourself. So, users would get redirected to the NCDC
>>>>>> server and access the data through the NCDC server. So, as things
>>>>>> stand, NCDC would have to add the WCS service.
>>>>>>
>>>>>> You could point to each NCDC dataset with an NcML wrapper and
>>>>>> actually serve the data from your server rather than redirecting
>>>>>> to NCDC. If you did that, you could add the WCS service. However,
>>>>>> the performance wouldn't be as good as direct from NCDC.
>>>>>>
>>>>>> I'm afraid I don't know about the WCS request for a time range.
>>>>>> So, I'll leave that for others to answer.
>>>>>>
>>>>>> Ethan
>>>>>>
>>>>>> Roy Mendelssohn wrote:
>>>>>>> Hi Ethan:
>>>>>>>
>>>>>>> I am not certain I follow. To be more specific, look at:
>>>>>>>
>>>>>>> http://oceanwatch.pfeg.noaa.gov/thredds/catalog.html
>>>>>>>
>>>>>>> and scroll down to the datasets " Remote test to Nomads THREDDS
>>>>>>> catalog", which is actually on a THREDDS server on an NCDC
>>>>>>> computer. They are remotely served through catalogs. Can we add
>>>>>>> the WCS service on our end, or will it only work if NCDC adds the
>>>>>>> WCS service.
>>>>>>>
>>>>>>> While I have your attention, I have not been able to get a
>>>>>>> straight answer if there is a way to send the WCS request to get
>>>>>>> a time range, rather than a single time, and if so what is the
>>>>>>> syntax for that.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> -Roy
>>>>>>>
>>>>>>> On Jul 6, 2007, at 11:24 AM, Ethan Davis wrote:
>>>>>>>
>>>>>>>> Hi Roy,
>>>>>>>>
>>>>>>>> If you are actually serving the dataset through your server
>>>>>>>> (which it sounds like you are since you mention existing NcML
>>>>>>>> for the dataset) rather than providing a link in your catalog to
>>>>>>>> the remote server, I believe you can simply add the WCS service
>>>>>>>> to your TDS configuration catalog and serve the data via WCS. It
>>>>>>>> shouldn't require any changes to the NcML, just to the service
>>>>>>>> element in the catalog.
>>>>>>>>
>>>>>>>> Of course, this depends on the dataset meeting the WCS server
>>>>>>>> requirements, e.g., evenly spaced grid.
>>>>>>>>
>>>>>>>> I'm not sure how it would perform. But I'll leave it for John to
>>>>>>>> respond on that.
>>>>>>>>
>>>>>>>> Ethan
>>>>>>>>
>>>>>>>> Roy Mendelssohn wrote:
>>>>>>>>> Hi:
>>>>>>>>>
>>>>>>>>> The question for today is if our TDS is "serving" a dataset
>>>>>>>>> remotely (ie. the remote site has their own TDS and we just
>>>>>>>>> link indirectly to it), can we add a service to our version
>>>>>>>>> that they don't have (just like we can aggregate a remote
>>>>>>>>> dataset that is not aggregated). In particular the remote
>>>>>>>>> dataset only has an OPeNDAP response, but we would like to be
>>>>>>>>> able to respond to both an OPeNDAP and a WCS response.
>>>>>>>>>
>>>>>>>>> If we change our NcML appropriately, will that work?
>>>>>>>>>
>>>>>>>>> Thanks in advance,
>>>>>>>>>
>>>>>>>>> -Roy
>>>>>>>>>
>>>>>>>>> **********************
>>>>>>>>> "The contents of this message do not reflect any position of
>>>>>>>>> the U.S. Government or NOAA."
>>>>>>>>> **********************
>>>>>>>>> Roy Mendelssohn
>>>>>>>>> Supervisory Operations Research Analyst
>>>>>>>>> NOAA/NMFS
>>>>>>>>> Environmental Research Division Southwest Fisheries Science
>>>>>>>>> Center
>>>>>>>>> 1352 Lighthouse Avenue
>>>>>>>>> Pacific Grove, CA 93950-2097
>>>>>>>>>
>>>>>>>>> e-mail: Roy.Mendelssohn at noaa.gov (Note new e-mail address)
>>>>>>>>> voice: (831)-648-9029
>>>>>>>>> fax: (831)-648-8440
>>>>>>>>> www: http://www.pfeg.noaa.gov/
>>>>>>>>>
>>>>>>>>> "Old age and treachery will overcome youth and skill."
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ===============================================================================
>>>>>>>>>
>>>>>>>>> To unsubscribe thredds, visit:
>>>>>>>>> http://www.unidata.ucar.edu/mailing-list-delete-form.html
>>>>>>>>> ===============================================================================
>>>>>>>>>
>>>>>>>>
>>>>>>>> --Ethan R. Davis Telephone: (303)
>>>>>>>> 497-8155
>>>>>>>> Software Engineer Fax: (303)
>>>>>>>> 497-8690
>>>>>>>> UCAR Unidata Program Center E-mail:
>>>>>>>> edavis at ucar.edu
>>>>>>>> P.O. Box 3000
>>>>>>>> Boulder, CO 80307-3000
>>>>>>>> http://www.unidata.ucar.edu/
>>>>>>>> ---------------------------------------------------------------------------
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> **********************
>>>>>>> "The contents of this message do not reflect any position of the
>>>>>>> U.S. Government or NOAA."
>>>>>>> **********************
>>>>>>> Roy Mendelssohn
>>>>>>> Supervisory Operations Research Analyst
>>>>>>> NOAA/NMFS
>>>>>>> Environmental Research Division Southwest Fisheries Science Center
>>>>>>> 1352 Lighthouse Avenue
>>>>>>> Pacific Grove, CA 93950-2097
>>>>>>>
>>>>>>> e-mail: Roy.Mendelssohn at noaa.gov (Note new e-mail address)
>>>>>>> voice: (831)-648-9029
>>>>>>> fax: (831)-648-8440
>>>>>>> www: http://www.pfeg.noaa.gov/
>>>>>>>
>>>>>>> "Old age and treachery will overcome youth and skill."
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>> --Glenn K. Rutledge
>>>>> Services Team Leader
>>>>> Remote Sensing and Applications Division
>>>>> NOMADS Project Manager
>>>>> National Oceanic and Atmospheric Administration
>>>>> National Climatic Data Center
>>>>> Asheville NC 28801
>>>>> Phone: (828) 271-4097
>>>>> Fax: (828) 271-4328
>>>>>
>>>>> NOMADS: http://nomads.ncdc.noaa.gov/
>>>>>
>>>>
>>>> **********************
>>>> "The contents of this message do not reflect any position of the
>>>> U.S. Government or NOAA."
>>>> **********************
>>>> Roy Mendelssohn
>>>> Supervisory Operations Research Analyst
>>>> NOAA/NMFS
>>>> Environmental Research Division Southwest Fisheries Science Center
>>>> 1352 Lighthouse Avenue
>>>> Pacific Grove, CA 93950-2097
>>>>
>>>> e-mail: Roy.Mendelssohn at noaa.gov (Note new e-mail address)
>>>> voice: (831)-648-9029
>>>> fax: (831)-648-8440
>>>> www: http://www.pfeg.noaa.gov/
>>>>
>>>> "Old age and treachery will overcome youth and skill."
>>>>
>>>>
>>>>
>>>> ===============================================================================
>>>>
>>>> To unsubscribe thredds, visit:
>>>> http://www.unidata.ucar.edu/mailing-list-delete-form.html
>>>> ===============================================================================
>>>>
>>> ===============================================================================
>>> To unsubscribe thredds, visit:
>>> http://www.unidata.ucar.edu/mailing-list-delete-form.html
>>> ===============================================================================
>>>
>
> **********************
> "The contents of this message do not reflect any position of the U.S.
> Government or NOAA."
> **********************
> Roy Mendelssohn
> Supervisory Operations Research Analyst
> NOAA/NMFS
> Environmental Research Division
> Southwest Fisheries Science Center
> 1352 Lighthouse Avenue
> Pacific Grove, CA 93950-2097
>
> e-mail: Roy.Mendelssohn at noaa.gov (Note new e-mail address)
> voice: (831)-648-9029
> fax: (831)-648-8440
> www: http://www.pfeg.noaa.gov/
>
> "Old age and treachery will overcome youth and skill."
>
>
==============================================================================
To unsubscribe thredds, visit:
http://www.unidata.ucar.edu/mailing-list-delete-form.html
==============================================================================
More information about the Thredds
mailing list