Re: [thredds] XML question for WCS THREDDS catalogs

Hi Nathan,

The problem is that the plus sign ("+") in the "dataset" query parameter gets un-url-encoded by the server into a space (" "). And there isn't a dataset ID that matches the unencoded string. I've made a fix to our catalog HTML to encode plus signs in the query parameter as "%2B" which fixes the problem.

The fix is in 3.16.22 on our FTP site:

ftp://ftp.unidata.ucar.edu/pub/thredds/3.16/

Ethan

Nathan Potter wrote:


I added the dataset to my catalog.xml file.

I pointed my browser at at:

    http://localhost:8080/opendap/catalog.xml

and clicked on the link to the dataset:

http://localhost:8080/opendap/catalog.html?dataset=wcs/DataFed/NetCDF/CCEXTTAU-INTEX.Organic+Black.Carbon.Aerosol


And (as Jobs would say) BOOM!


N


On Oct 24, 2007, at 12:01 PM, Ethan Davis wrote:

Hi Nathan, Dan,

I was just looking at this (John forwarded me a note from Dan). I'm not seeing the NullPointerException but I am seeing some problems with encoding of the "+" in URLs. What are you doing when you get the NullPointerException?

Ethan

Nathan Potter wrote:

Ethan and John,


We are having a problem with (what we think is) a (valid) THREDDS catalog entry causing the THREDDS code to throw a null pointer exception in. (See the included emails for the background.)

The catalog entry is:


    <dataset name="CCEXTTAU-INTEX.Organic+Black.Carbon.Aerosol"
urlPath="wcs/DataFed/NetCDF/CCEXTTAU-INTEX.Organic+Black.Carbon.Aerosol" ID="wcs/DataFed/NetCDF/CCEXTTAU-INTEX.Organic+Black.Carbon.Aerosol" > <property name="wcs-request" value="http://webapps.datafed.net/ogc_NASA.wsfl?SERVICE=WCS&amp;REQUEST=GetCoverage&amp;VERSION=1.0.0&amp;CRS=EPSG:4326&amp;COVERAGE=GOCART_G_OL.CCEXTTAU&amp;TIME=2007-06-24T23:00:00&amp;BBOX=-180,-90,180,90,0,0&amp;WIDTH=-1&amp;HEIGHT=-1&amp;DEPTH=-1&amp;FORMAT=NetCDF"; />
        <property name="returnType" value="netCDF" />
        <property name="cacheTime" value="600" />
        <serviceName>OPeNDAP-Hyrax</serviceName>
   </dataset>



The stack trace that I get is:

java.lang.NullPointerException
at thredds.servlet.CatalogServicesServlet.handleCatalogServiceRequest(CatalogServicesServlet.java:226) at thredds.servlet.DataRootHandler.processReqForCatalog(DataRootHandler.java:1117) at opendap.bes.ThreddsDispatchHandler.handleRequest(ThreddsDispatchHandler.java:232) at opendap.coreServlet.DispatchServlet.doGet(DispatchServlet.java:437)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)


And it appears to a problem ONLY when there is a plus sign ("+") in the value of the ID attribute.


Any ideas?


N








Begin forwarded message:

From: Daniel Holloway <d.holloway@xxxxxxxxxxx>
Date: October 23, 2007 11:01:37 AM PDT
To: Nathan Potter <ndp@xxxxxxxxxxx>
Subject: Re: XML question for WCS THREDDS catalogs


On Oct 23, 2007, at 1:53 PM, Nathan Potter wrote:


On Oct 23, 2007, at 10:01 AM, Daniel Holloway wrote:


On Oct 23, 2007, at 12:21 PM, Nathan Potter wrote:


Dan,

Several things strike me as potential problems:

1) The Exception is being thrown deep in the THREDDS code. Disturbingly, it appears to be in a servlert implmentation that I am not running in Hyrax. Are you running a TDS too?

   Hi Nathan,

Thanks for the quick reply. No, I'm not running a TDS on this platform as well.


2) A plus sign in a URL (outside of the constraint expression) mught get filtered by the new security features in the PLFS, and this would cause trouble!

It's not a big problem for me, I was more interested in knowing where the problem existed and why. If these characters aren't allowable in XML, or THREDDS, then I need to document that in the configuration guide for the handler. It seems to me that these are valid characters, or at least should be able to be replaced with '&plus' without it causing these problems.

    Anyway, welcome back and thanks for the reply.

       Dan


p.s. Should we enter a ticket on this to keep track of the problem?


I think so. I am still pretty non functional from jet lag.... I'll be on my game tomorrow and I'll try to tack it down. Will you send me the THREDDS catalog entry that caused the error?


   Nathan,

Here's the dataset element that is causing the problem. The best bet is to replace the element in your example catalog.xml with this one. The problem seems to be the '+' sign in the ID attribute. I've experimented and found that it only fails when the '+' sign is in the ID attribute, no idea why.

--------
<dataset name="CCEXTTAU-INTEX.Organic+Black.Carbon.Aerosol" urlPath="wcs/DataFed/NetCDF/CCEXTTAU-INTEX.Organic+Black.Carbon.Aerosol" ID="wcs/DataFed/NetCDF/CCEXTTAU-INTEX.Organic+Black.Carbon.Aerosol" > <property name="wcs-request" value="http://webapps.datafed.net/ogc_NASA.wsfl?SERVICE=WCS&amp;REQUEST=GetCoverage&amp;VERSION=1.0.0&amp;CRS=EPSG:4326&amp;COVERAGE=GOCART_G_OL.CCEXTTAU&amp;TIME=2007-06-24T23:00:00&amp;BBOX=-180,-90,\ 180,90,0,0&amp;WIDTH=-1&amp;HEIGHT=-1&amp;DEPTH=-1&amp;FORMAT=NetCDF" />
                <property name="returnType" value="netCDF" />
                <property name="cacheTime" value="600" />
                <serviceName>OPeNDAP-Hyrax</serviceName>
           </dataset>

-------

   Dan

N




N






On Oct 23, 2007, at 8:52 AM, Daniel Holloway wrote:

Nathan,

I've been playing with the configuration of a Hyrax server using the latest OLFS/BES code. There is/was a dataset that has a '+' sign in the name. I get an error from the OLFS when using this string in the ID attribute of the dataset element. As follows:
------
<dataset name="CCEXTTAU-INTEX.Organic+Black.Carbon.Aerosol" urlPath="wcs/DataFed/NetCDF/CCEXTTAU-INTEX.Organic+Black.Carbon.Aerosol" ID="wcs/DataFed/NetCDF/CCEXTTAU-INTEX.Organic+Black.Carbon.Aerosol" > <property name="wcs-request" value="http://webapps.datafed.net/ogc_NASA.wsfl?SERVICE=WCS&amp;REQUEST=GetCoverage&amp;VERSION=1.0.0&amp;CRS=EPSG:4326&amp;COVERAGE=GOCART_G_OL.CCEXTTAU&amp;TIME=2007-06-24T23:00:00&amp;BBOX=-180,-90,\ 180,90,0,0&amp;WIDTH=-1&amp;HEIGHT=-1&amp;DEPTH=-1&amp;FORMAT=NetCDF" />
                <property name="returnType" value="netCDF" />
                <property name="cacheTime" value="600" />
                <serviceName>OPeNDAP-Hyrax</serviceName>
           </dataset>
-------

   The error is:
-------
Error { code = -1; message = "java.lang.NullPointerException: null [CatalogServicesServlet.java - line 226]"; };
-------

I only get an error if the '+' sign is used in the 'ID' attribute, the others don't seem to make any difference. I've tried replacing the '+' with a &plus; but that breaks the server's access to the catalog completely. Not exactly sure why but if I use a &plus; in the XML I get a 'Resource Not Found' response from the server when trying to open the catalog.

  Anyway, figured I'd ask before submitting a Trac ticket.

  Dan


= = =
Nathan Potter                        ndp at opendap.org
OPeNDAP, Inc.                        541.752.1852



= = =
Nathan Potter                        ndp at opendap.org
OPeNDAP, Inc.                        541.752.1852



= = =
Nathan Potter                        ndp at opendap.org
OPeNDAP, Inc.                        541.752.1852


--
Ethan R. Davis                                Telephone: (303) 497-8155
Software Engineer                             Fax:       (303) 497-8690
UCAR Unidata Program Center                   E-mail:    edavis@xxxxxxxx
P.O. Box 3000
Boulder, CO 80307-3000 http://www.unidata.ucar.edu/ ---------------------------------------------------------------------------



= = =
Nathan Potter                        ndp at opendap.org
OPeNDAP, Inc.                        541.752.1852


--
Ethan R. Davis                                Telephone: (303) 497-8155
Software Engineer                             Fax:       (303) 497-8690
UCAR Unidata Program Center                   E-mail:    edavis@xxxxxxxx
P.O. Box 3000
Boulder, CO  80307-3000                       http://www.unidata.ucar.edu/
---------------------------------------------------------------------------




  • 2007 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: