[thredds] XML question for WCS THREDDS catalogs
Ethan Davis
edavis at unidata.ucar.edu
Wed Oct 24 17:12:48 MDT 2007
Hi Nathan,
The problem is that the plus sign ("+") in the "dataset" query parameter
gets un-url-encoded by the server into a space (" "). And there isn't a
dataset ID that matches the unencoded string. I've made a fix to our
catalog HTML to encode plus signs in the query parameter as "%2B" which
fixes the problem.
The fix is in 3.16.22 on our FTP site:
ftp://ftp.unidata.ucar.edu/pub/thredds/3.16/
Ethan
Nathan Potter wrote:
>
>
> I added the dataset to my catalog.xml file.
>
> I pointed my browser at at:
>
> http://localhost:8080/opendap/catalog.xml
>
> and clicked on the link to the dataset:
>
> http://localhost:8080/opendap/catalog.html?dataset=wcs/DataFed/NetCDF/CCEXTTAU-INTEX.Organic+Black.Carbon.Aerosol
>
>
>
> And (as Jobs would say) BOOM!
>
>
> N
>
>
> On Oct 24, 2007, at 12:01 PM, Ethan Davis wrote:
>
>> Hi Nathan, Dan,
>>
>> I was just looking at this (John forwarded me a note from Dan). I'm
>> not seeing the NullPointerException but I am seeing some problems
>> with encoding of the "+" in URLs. What are you doing when you get the
>> NullPointerException?
>>
>> Ethan
>>
>> Nathan Potter wrote:
>>>
>>> Ethan and John,
>>>
>>>
>>> We are having a problem with (what we think is) a (valid) THREDDS
>>> catalog entry causing the THREDDS code to throw a null pointer
>>> exception in. (See the included emails for the background.)
>>>
>>> The catalog entry is:
>>>
>>>
>>> <dataset name="CCEXTTAU-INTEX.Organic+Black.Carbon.Aerosol"
>>>
>>> urlPath="wcs/DataFed/NetCDF/CCEXTTAU-INTEX.Organic+Black.Carbon.Aerosol"
>>>
>>>
>>> ID="wcs/DataFed/NetCDF/CCEXTTAU-INTEX.Organic+Black.Carbon.Aerosol" >
>>> <property name="wcs-request"
>>> value="http://webapps.datafed.net/ogc_NASA.wsfl?SERVICE=WCS&REQUEST=GetCoverage&VERSION=1.0.0&CRS=EPSG:4326&COVERAGE=GOCART_G_OL.CCEXTTAU&TIME=2007-06-24T23:00:00&BBOX=-180,-90,180,90,0,0&WIDTH=-1&HEIGHT=-1&DEPTH=-1&FORMAT=NetCDF"
>>> />
>>> <property name="returnType" value="netCDF" />
>>> <property name="cacheTime" value="600" />
>>> <serviceName>OPeNDAP-Hyrax</serviceName>
>>> </dataset>
>>>
>>>
>>>
>>> The stack trace that I get is:
>>>
>>> java.lang.NullPointerException
>>> at
>>> thredds.servlet.CatalogServicesServlet.handleCatalogServiceRequest(CatalogServicesServlet.java:226)
>>>
>>> at
>>> thredds.servlet.DataRootHandler.processReqForCatalog(DataRootHandler.java:1117)
>>>
>>> at
>>> opendap.bes.ThreddsDispatchHandler.handleRequest(ThreddsDispatchHandler.java:232)
>>>
>>> at
>>> opendap.coreServlet.DispatchServlet.doGet(DispatchServlet.java:437)
>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
>>>
>>>
>>> And it appears to a problem ONLY when there is a plus sign ("+") in
>>> the value of the ID attribute.
>>>
>>>
>>> Any ideas?
>>>
>>>
>>> N
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Begin forwarded message:
>>>
>>>> From: Daniel Holloway <d.holloway at opendap.org>
>>>> Date: October 23, 2007 11:01:37 AM PDT
>>>> To: Nathan Potter <ndp at opendap.org>
>>>> Subject: Re: XML question for WCS THREDDS catalogs
>>>>
>>>>
>>>> On Oct 23, 2007, at 1:53 PM, Nathan Potter wrote:
>>>>
>>>>>
>>>>> On Oct 23, 2007, at 10:01 AM, Daniel Holloway wrote:
>>>>>
>>>>>>
>>>>>> On Oct 23, 2007, at 12:21 PM, Nathan Potter wrote:
>>>>>>
>>>>>>>
>>>>>>> Dan,
>>>>>>>
>>>>>>> Several things strike me as potential problems:
>>>>>>>
>>>>>>> 1) The Exception is being thrown deep in the THREDDS code.
>>>>>>> Disturbingly, it appears to be in a servlert implmentation that
>>>>>>> I am not running in Hyrax. Are you running a TDS too?
>>>>>>
>>>>>> Hi Nathan,
>>>>>>
>>>>>> Thanks for the quick reply. No, I'm not running a TDS on
>>>>>> this platform as well.
>>>>>>
>>>>>>>
>>>>>>> 2) A plus sign in a URL (outside of the constraint expression)
>>>>>>> mught get filtered by the new security features in the PLFS, and
>>>>>>> this would cause trouble!
>>>>>>>
>>>>>> It's not a big problem for me, I was more interested in
>>>>>> knowing where the problem existed and why. If these characters
>>>>>> aren't allowable in XML, or THREDDS, then I need to document that
>>>>>> in the configuration guide for the handler. It seems to me that
>>>>>> these are valid characters, or at least should be able to be
>>>>>> replaced with '&plus' without it causing these problems.
>>>>>>
>>>>>> Anyway, welcome back and thanks for the reply.
>>>>>>
>>>>>> Dan
>>>>>>
>>>>>>
>>>>>> p.s. Should we enter a ticket on this to keep track of the
>>>>>> problem?
>>>>>
>>>>>
>>>>> I think so. I am still pretty non functional from jet lag.... I'll
>>>>> be on my game tomorrow and I'll try to tack it down. Will you send
>>>>> me the THREDDS catalog entry that caused the error?
>>>>>
>>>>
>>>> Nathan,
>>>>
>>>> Here's the dataset element that is causing the problem. The
>>>> best bet is to replace the element in your example catalog.xml with
>>>> this one. The problem seems to be the '+' sign in the ID
>>>> attribute. I've experimented and found that it only fails when the
>>>> '+' sign is in the ID attribute, no idea why.
>>>>
>>>> --------
>>>> <dataset
>>>> name="CCEXTTAU-INTEX.Organic+Black.Carbon.Aerosol"
>>>>
>>>> urlPath="wcs/DataFed/NetCDF/CCEXTTAU-INTEX.Organic+Black.Carbon.Aerosol"
>>>>
>>>>
>>>> ID="wcs/DataFed/NetCDF/CCEXTTAU-INTEX.Organic+Black.Carbon.Aerosol" >
>>>> <property name="wcs-request"
>>>> value="http://webapps.datafed.net/ogc_NASA.wsfl?SERVICE=WCS&REQUEST=GetCoverage&VERSION=1.0.0&CRS=EPSG:4326&COVERAGE=GOCART_G_OL.CCEXTTAU&TIME=2007-06-24T23:00:00&BBOX=-180,-90,\
>>>>
>>>> 180,90,0,0&WIDTH=-1&HEIGHT=-1&DEPTH=-1&FORMAT=NetCDF"
>>>> />
>>>> <property name="returnType" value="netCDF" />
>>>> <property name="cacheTime" value="600" />
>>>> <serviceName>OPeNDAP-Hyrax</serviceName>
>>>> </dataset>
>>>>
>>>> -------
>>>>
>>>> Dan
>>>>
>>>>> N
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>> N
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Oct 23, 2007, at 8:52 AM, Daniel Holloway wrote:
>>>>>>>
>>>>>>>> Nathan,
>>>>>>>>
>>>>>>>> I've been playing with the configuration of a Hyrax server
>>>>>>>> using the latest OLFS/BES code. There is/was a dataset that
>>>>>>>> has a '+' sign in the name. I get an error from the OLFS when
>>>>>>>> using this string in the ID attribute of the dataset element.
>>>>>>>> As follows:
>>>>>>>> ------
>>>>>>>> <dataset
>>>>>>>> name="CCEXTTAU-INTEX.Organic+Black.Carbon.Aerosol"
>>>>>>>>
>>>>>>>> urlPath="wcs/DataFed/NetCDF/CCEXTTAU-INTEX.Organic+Black.Carbon.Aerosol"
>>>>>>>>
>>>>>>>>
>>>>>>>> ID="wcs/DataFed/NetCDF/CCEXTTAU-INTEX.Organic+Black.Carbon.Aerosol"
>>>>>>>> >
>>>>>>>> <property name="wcs-request"
>>>>>>>> value="http://webapps.datafed.net/ogc_NASA.wsfl?SERVICE=WCS&REQUEST=GetCoverage&VERSION=1.0.0&CRS=EPSG:4326&COVERAGE=GOCART_G_OL.CCEXTTAU&TIME=2007-06-24T23:00:00&BBOX=-180,-90,\
>>>>>>>>
>>>>>>>> 180,90,0,0&WIDTH=-1&HEIGHT=-1&DEPTH=-1&FORMAT=NetCDF"
>>>>>>>> />
>>>>>>>> <property name="returnType" value="netCDF" />
>>>>>>>> <property name="cacheTime" value="600" />
>>>>>>>> <serviceName>OPeNDAP-Hyrax</serviceName>
>>>>>>>> </dataset>
>>>>>>>> -------
>>>>>>>>
>>>>>>>> The error is:
>>>>>>>> -------
>>>>>>>> Error { code = -1; message = "java.lang.NullPointerException:
>>>>>>>> null [CatalogServicesServlet.java - line 226]"; };
>>>>>>>> -------
>>>>>>>>
>>>>>>>> I only get an error if the '+' sign is used in the 'ID'
>>>>>>>> attribute, the others don't seem to make any difference. I've
>>>>>>>> tried replacing the '+' with a + but that breaks the
>>>>>>>> server's access to the catalog completely. Not exactly sure
>>>>>>>> why but if I use a + in the XML I get a 'Resource Not
>>>>>>>> Found' response from the server when trying to open the catalog.
>>>>>>>>
>>>>>>>> Anyway, figured I'd ask before submitting a Trac ticket.
>>>>>>>>
>>>>>>>> Dan
>>>>>>>>
>>>>>>>
>>>>>>> = = =
>>>>>>> Nathan Potter ndp at opendap.org
>>>>>>> OPeNDAP, Inc. 541.752.1852
>>>>>>>
>>>>>>
>>>>>
>>>>> = = =
>>>>> Nathan Potter ndp at opendap.org
>>>>> OPeNDAP, Inc. 541.752.1852
>>>>>
>>>>
>>>
>>> = = =
>>> Nathan Potter ndp at opendap.org
>>> OPeNDAP, Inc. 541.752.1852
>>>
>>
>> --
>> Ethan R. Davis Telephone: (303) 497-8155
>> Software Engineer Fax: (303) 497-8690
>> UCAR Unidata Program Center E-mail: edavis at ucar.edu
>> P.O. Box 3000
>> Boulder, CO 80307-3000
>> http://www.unidata.ucar.edu/
>> ---------------------------------------------------------------------------
>>
>>
>>
>
> = = =
> Nathan Potter ndp at opendap.org
> OPeNDAP, Inc. 541.752.1852
>
--
Ethan R. Davis Telephone: (303) 497-8155
Software Engineer Fax: (303) 497-8690
UCAR Unidata Program Center E-mail: edavis at ucar.edu
P.O. Box 3000
Boulder, CO 80307-3000 http://www.unidata.ucar.edu/
---------------------------------------------------------------------------
More information about the thredds
mailing list