Adding a service to a remotely served dataset
Ethan Davis
edavis at unidata.ucar.edu
Mon Jul 9 13:53:23 MDT 2007
John,
Is this capability available with the netCDF subset service as well?
Ethan
John Caron wrote:
> We made an extension of the WCS Dataset URL format to allow the TDS to
> serve remote datasets. Identify the dataset by adding the parameter
> dataset whose value is a URL:
>
> http://servername:8080/thredds/wcs?dataset=datasetURL&
>
> The URL must be a dataset readable by the NetCDF-Java library,
> typically an OPeNDAP dataset on another server. It must have gridded
> data, with identifiable coordinate systems, etc. For example, an
> OPeNDAP URL might be
>
> http://las.pfeg.noaa.gov/cgi-bin/nph-dods/data/oceanwatch/nrt/gac/AG14day.nc
>
> This can be served remotely as a WCS dataset with this URL:
>
> http://servername:8080/thredds/wcs?dataset=http://las.pfeg.noaa.gov/cgi-bin/nph-dods/data/oceanwatch/nrt/gac/AG14day.nc&
>
>
>
> This is non-standard WCS; we havent emphasized this feature until we
> have a chance to review security implications. The WCS service now if
> off by default, and must explicitly be turned on in threddsConfig.xml,
> mostly because of this feature. We should probably seperately allow
> users to enable/disable "remote WCS access".
>
>
> Oliver Newell wrote:
>> Hi Roy -
>>
>> FYI - We're currently experimenting with implementing WCS services,
>> and the same requirement (wish?) as what you're describing has been
>> raised. Basically, the need to discover a data set via a cataloging
>> mechanism (THREDDS, an OGC catalog, a DOD NCES catalog) and then be
>> able to access it via a WCS that resides somewhere on the network. My
>> initial take on it is that since WCS data access requests work with
>> unique identifiers (not URLs), this limits the capabilities of the
>> service in the sense that the Id-to-resource (data file or db)
>> mapping is managed locally by the WCS. In other words, the WCS has to
>> 'know about' the resource in advance of the request, in order to do
>> the mapping.
>>
>> I think the WCS service should support a mode of operation where a
>> resource's URL is included in the requests to the service, in order
>> to support the external WCS concept. I think it is still important
>> to maintain the Id as a separate entity, so it is not just a question
>> of using a URL for the coverage identifier, though that idea might be
>> tempting.
>>
>> The idea needs flushing out a bit, but it is still early enough in
>> WCS's development that changes like this can be incorporated into the
>> spec via the OGC working group (my opinion). It definitely seems like
>> it would be a useful capability to address the situation you describe.
>>
>> -Oliver
>>
>> Roy Mendelssohn wrote:
>>> Hi Glenn:
>>>
>>> No problem. But we were interested in the general problem, as there
>>> are other datasets which also do not have WCS and which we would
>>> like to add it, rather than trying to get each of them to add it
>>> (Some just have an OPeNDAP server, not a THREDDS server). Yours
>>> allowed me to give Ethan a concrete example so he could look at the
>>> catalogs.
>>>
>>> Thanks,
>>>
>>> -Roy
>>> On Jul 9, 2007, at 7:52 AM, Glenn.Rutledge wrote:
>>>
>>>> Hello Roy-
>>>> That particular dataset will be turned on shortly via WCS. That
>>>> was an oversight. Nice to have users telling us where we need
>>>> stuff! Glenn
>>>>
>>>> Ethan Davis wrote the following on 7/6/2007 6:02 PM:
>>>>> Hi Roy,
>>>>>
>>>>> Looks like you are referencing NCDC catalogs rather than serving
>>>>> the datasets yourself. So, users would get redirected to the NCDC
>>>>> server and access the data through the NCDC server. So, as things
>>>>> stand, NCDC would have to add the WCS service.
>>>>>
>>>>> You could point to each NCDC dataset with an NcML wrapper and
>>>>> actually serve the data from your server rather than redirecting
>>>>> to NCDC. If you did that, you could add the WCS service. However,
>>>>> the performance wouldn't be as good as direct from NCDC.
>>>>>
>>>>> I'm afraid I don't know about the WCS request for a time range.
>>>>> So, I'll leave that for others to answer.
>>>>>
>>>>> Ethan
>>>>>
>>>>> Roy Mendelssohn wrote:
>>>>>> Hi Ethan:
>>>>>>
>>>>>> I am not certain I follow. To be more specific, look at:
>>>>>>
>>>>>> http://oceanwatch.pfeg.noaa.gov/thredds/catalog.html
>>>>>>
>>>>>> and scroll down to the datasets " Remote test to Nomads THREDDS
>>>>>> catalog", which is actually on a THREDDS server on an NCDC
>>>>>> computer. They are remotely served through catalogs. Can we add
>>>>>> the WCS service on our end, or will it only work if NCDC adds the
>>>>>> WCS service.
>>>>>>
>>>>>> While I have your attention, I have not been able to get a
>>>>>> straight answer if there is a way to send the WCS request to get
>>>>>> a time range, rather than a single time, and if so what is the
>>>>>> syntax for that.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> -Roy
>>>>>>
>>>>>> On Jul 6, 2007, at 11:24 AM, Ethan Davis wrote:
>>>>>>
>>>>>>> Hi Roy,
>>>>>>>
>>>>>>> If you are actually serving the dataset through your server
>>>>>>> (which it sounds like you are since you mention existing NcML
>>>>>>> for the dataset) rather than providing a link in your catalog to
>>>>>>> the remote server, I believe you can simply add the WCS service
>>>>>>> to your TDS configuration catalog and serve the data via WCS. It
>>>>>>> shouldn't require any changes to the NcML, just to the service
>>>>>>> element in the catalog.
>>>>>>>
>>>>>>> Of course, this depends on the dataset meeting the WCS server
>>>>>>> requirements, e.g., evenly spaced grid.
>>>>>>>
>>>>>>> I'm not sure how it would perform. But I'll leave it for John to
>>>>>>> respond on that.
>>>>>>>
>>>>>>> Ethan
>>>>>>>
>>>>>>> Roy Mendelssohn wrote:
>>>>>>>> Hi:
>>>>>>>>
>>>>>>>> The question for today is if our TDS is "serving" a dataset
>>>>>>>> remotely (ie. the remote site has their own TDS and we just
>>>>>>>> link indirectly to it), can we add a service to our version
>>>>>>>> that they don't have (just like we can aggregate a remote
>>>>>>>> dataset that is not aggregated). In particular the remote
>>>>>>>> dataset only has an OPeNDAP response, but we would like to be
>>>>>>>> able to respond to both an OPeNDAP and a WCS response.
>>>>>>>>
>>>>>>>> If we change our NcML appropriately, will that work?
>>>>>>>>
>>>>>>>> Thanks in advance,
>>>>>>>>
>>>>>>>> -Roy
>>>>>>>>
>>>>>>>> **********************
>>>>>>>> "The contents of this message do not reflect any position of
>>>>>>>> the U.S. Government or NOAA."
>>>>>>>> **********************
>>>>>>>> Roy Mendelssohn
>>>>>>>> Supervisory Operations Research Analyst
>>>>>>>> NOAA/NMFS
>>>>>>>> Environmental Research Division Southwest Fisheries Science
>>>>>>>> Center
>>>>>>>> 1352 Lighthouse Avenue
>>>>>>>> Pacific Grove, CA 93950-2097
>>>>>>>>
>>>>>>>> e-mail: Roy.Mendelssohn at noaa.gov (Note new e-mail address)
>>>>>>>> voice: (831)-648-9029
>>>>>>>> fax: (831)-648-8440
>>>>>>>> www: http://www.pfeg.noaa.gov/
>>>>>>>>
>>>>>>>> "Old age and treachery will overcome youth and skill."
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ===============================================================================
>>>>>>>>
>>>>>>>> To unsubscribe thredds, visit:
>>>>>>>> http://www.unidata.ucar.edu/mailing-list-delete-form.html
>>>>>>>> ===============================================================================
>>>>>>>>
>>>>>>>
>>>>>>> --Ethan R. Davis Telephone: (303)
>>>>>>> 497-8155
>>>>>>> Software Engineer Fax: (303)
>>>>>>> 497-8690
>>>>>>> UCAR Unidata Program Center E-mail:
>>>>>>> edavis at ucar.edu
>>>>>>> P.O. Box 3000
>>>>>>> Boulder, CO 80307-3000
>>>>>>> http://www.unidata.ucar.edu/
>>>>>>> ---------------------------------------------------------------------------
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> **********************
>>>>>> "The contents of this message do not reflect any position of the
>>>>>> U.S. Government or NOAA."
>>>>>> **********************
>>>>>> Roy Mendelssohn
>>>>>> Supervisory Operations Research Analyst
>>>>>> NOAA/NMFS
>>>>>> Environmental Research Division Southwest Fisheries Science Center
>>>>>> 1352 Lighthouse Avenue
>>>>>> Pacific Grove, CA 93950-2097
>>>>>>
>>>>>> e-mail: Roy.Mendelssohn at noaa.gov (Note new e-mail address)
>>>>>> voice: (831)-648-9029
>>>>>> fax: (831)-648-8440
>>>>>> www: http://www.pfeg.noaa.gov/
>>>>>>
>>>>>> "Old age and treachery will overcome youth and skill."
>>>>>>
>>>>>>
>>>>>
>>>>
>>>> --Glenn K. Rutledge
>>>> Services Team Leader
>>>> Remote Sensing and Applications Division
>>>> NOMADS Project Manager
>>>> National Oceanic and Atmospheric Administration
>>>> National Climatic Data Center
>>>> Asheville NC 28801
>>>> Phone: (828) 271-4097
>>>> Fax: (828) 271-4328
>>>>
>>>> NOMADS: http://nomads.ncdc.noaa.gov/
>>>>
>>>
>>> **********************
>>> "The contents of this message do not reflect any position of the
>>> U.S. Government or NOAA."
>>> **********************
>>> Roy Mendelssohn
>>> Supervisory Operations Research Analyst
>>> NOAA/NMFS
>>> Environmental Research Division Southwest Fisheries Science Center
>>> 1352 Lighthouse Avenue
>>> Pacific Grove, CA 93950-2097
>>>
>>> e-mail: Roy.Mendelssohn at noaa.gov (Note new e-mail address)
>>> voice: (831)-648-9029
>>> fax: (831)-648-8440
>>> www: http://www.pfeg.noaa.gov/
>>>
>>> "Old age and treachery will overcome youth and skill."
>>>
>>>
>>>
>>> ===============================================================================
>>>
>>> To unsubscribe thredds, visit:
>>> http://www.unidata.ucar.edu/mailing-list-delete-form.html
>>> ===============================================================================
>>>
>>
>> ===============================================================================
>>
>> To unsubscribe thredds, visit:
>> http://www.unidata.ucar.edu/mailing-list-delete-form.html
>> ===============================================================================
>>
>
> ===============================================================================
>
> To unsubscribe thredds, visit:
> http://www.unidata.ucar.edu/mailing-list-delete-form.html
> ===============================================================================
>
--
Ethan R. Davis Telephone: (303) 497-8155
Software Engineer Fax: (303) 497-8690
UCAR Unidata Program Center E-mail: edavis at ucar.edu
P.O. Box 3000
Boulder, CO 80307-3000 http://www.unidata.ucar.edu/
---------------------------------------------------------------------------
==============================================================================
To unsubscribe thredds, visit:
http://www.unidata.ucar.edu/mailing-list-delete-form.html
==============================================================================
More information about the Thredds
mailing list